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I.  INTRODUCTION 
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sequence  is  horn,  in  ||“«sSeme£t  system  require-. 
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this  specification. 
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Figure  1.  Program  Reports. 


II.  DATA  ACQUISITION  SUBSYSTEM 


Data  Acquisition  Categories 

Combat-crew  training  performance  information  may  come  froir 
a number  of  sources  and  through  a variety  of  media.  The  data 
acquisition  subsystem  must  provide  the  capability  for  acquiring 
all  information  needed  for  measurement  and  the  interface  with 
manual  and  automatic  means  for  compiling  an  integrated, 
correlated,  common  raw  data  base. 

Data  sources  are  divided  into  five  categories  for  the 
current  treatment:  (1)  aircraft  or  simulator  data  collection, 

(2)  field  data  collection,  (3)  briefing  or  debriefing  data 
collection,  (4)  documentary  data  collection,  and  ( 5)  external 
data  sources.  Each  information  source  poses  different  conditi 
for  data  acquisition,  although  a form  of  recording  is  used 
each  case,  requiring  some  means  for  data  playback. 

The  Data  Acquisition  Subsystem,  therefore,  consists  of  the 
major  components  of  the  block  diagram  shown  in  Figure  2.  This 
chapter  will  be  devoted  to  specification  of  the  data  jollectio 
devices  for  each  source  of  information  (as  listed  in  Table  1) , 
and  devices  for  data  playback. 

Aircraft/Simulator  Data  Collection  Station  (ASDCS| 

It  was  found  through  design  tradeoff  studies,  that  training 
performance  measurement  inflight  or  in  simulator  solely  Y 
of  (l)  video  or  photo  recording  or  (2)  digital  recording , was 
not  entireli  satisfactory  for  either  technique.  a decisi°n 

must  be  made  to  go  one  way  or  the  other,  assuming  that  vide 
legibility  requirements  can  be  met,  then  a video  recordi  9 
system  would  be  chosen  for  comprehensive  measurement,  simplici  y 
and  flexibility  for  the  pursuit  of  research  9?als,  and  lower 
overall  costs.  However,  a hybrid  system  combining  the  adva^ages 
of  video/photo  and  digital  recording  is  preferable  to  an  unmixed 
svstem.  A hybrid  system  is  assumed  in  the  current  specification, 
St  it  is  also  required  that  the  video/photo  and  digital  record- 
ing  systems  have  a stand-alone  capability /thus  three  da  a 
acquisition  systems  are  defined:  (1)  a video/photo  recording 

system,  (2)  a digital  recording  system,  and  (3)  a hybrid  recor 

ing  system. 

The  distribution  of  measurement  parameters  for  recording  by 
video  or  digital  techniques  is  portrayed  in  Figure  3.  Audio 
information  must  be  collected  by  both  video/photo  and  digital 
recording  to  provide  a stand-alone  data  collection  capability 
with  either  technique;  consequently,  these  data  acquisition 
devices  will  be  termed  Audio/Video  Recording  (AVR)  and  Audio/ 
Digital  Recording  (ADR)  in  correspondence  with  previous  usage,. 
Most  pictorial  information  can  be  easily  obtained  only  with 


3 


AIRCRAFT/ 

SIMULATOR 

DATA 

COLLECTION 


FIELD 

UA1A 

COLLECTION 

EXTERNAL 

DATA 

COLLECTION 


BRIEFING/ 

DEBRIEFING 

DATA 

COLLECTION 


DOCUMENTARY 

DATA 

COLLECTION 


JDATA 

PLAYBACK 

STATION 


DATA 

PROCESSING 


Figure  2.  Data  Acquisition  Block  Diagram. 
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TABLE  1 


CATEGORIES  OF  INFORMATION  SOURCES  AND 
CORRESPONDING  DATA  COLLECTION  DEVICES 


SOURCE  OF 
INFORMATION 

DATA  COLLECTION 
DEVICES 

1. 

Aircraft/Simulator 

Audio/Video  Recorder  + Aux. 
Camera  + Audio/Digital 
Recorder 

2. 

Field  (e.g.,  Runway, 
GCA  Radar , Range ) 

Camera  + Transceivers  + Audio 
Recorder 

3. 

Briefing/Debriefing 

Audio  Recorder  + Slide  Camera 

4. 

Documents 

Manual  Conversion  from  Documents 
to  Paper  Tape  Necessary  (e.g., 
Dash-One,  Phase  Manuals) 

5. 

External  Data  (e.g.. 
Ground  Tracking  Radar, 
Re lated-Experiment 
Results) 

Data  Received  in  Computer- 
Compatible  Form  (Cards, 
Magnetic  Tape,  Paper  Tape) 
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video/photo  techniques,  therefore,  other  avionics  display 
information,  such  as  radar,  navigation  and  weapon  control,  will 
be  excluded  from  the  ADR  to  reduce  the  complexity  and  associated 
costs  of  attempting  to  record  large  numbers  of  parameters  through 
digital  methods.  Aircraft  and  control  parameters  (e.g.,  airspeed, 
altitude)  will  be  collected  on  the  ADR,  and  as  many  as  possible 
within  the  f ield-of-view  of  the  AVR,  In  a similar  manner, 
discrete  signals  (e.g.,  wheels  up,  speed  brakes  out)  will  be 
recorded  on  the  ADR,  and  means  will  be  developed  to  allow 
recording  a small  number  of  critical  discrete  parameters  (e.g., 
weapons  release)  on  the  AVR  to  permit  an  AVR  stand-alone 
capability  and  to  enhance  manual  measurement  processing. 


A block  diagram  of  the  Aircraft/Simulator  Data  Collection 
Station  is  shown  in  Figure  4.  An  Auxiliary  Camera  (AC)  is 
necessary  to  provide  information  where  neither  thd’  AVR  nor  ADR 
applicable;  such  a camera  is  specifically  needed  to  document 
position  over  recognizable  terrain  and  the  drop  zone  (Combat 
Airlift  Training),  but  is  likely  to  have  general  usefulness  for 
providing  data  recording  for  items  outside  the  view  of  the 
video  cameras.  Recording  controls  and  displays  (RCD)  are  need 
to  provide  manual  and  simple  programming  of  recording  events, 
for  display  of  information  for  equipment  s*et-up  and  for  the  use 
of  an  onboard  experimenter.  Interface  with  the  interphone  system 
for  audio  recording  completes  the  aircraft/simulator  data 
collection  station. 


It  is  recommended  that  the  same  data  acqusition  system  be 
used  in  both  aircraft  and  simulator  environments,  although  in 
some  simulator  applications  re-progranuning  of  the  simulation 
computer  may  provide  the  equivalent  of  the  specified  digital 
recording. 


Audio/Video  Recording  (AVR) . Figure  5 presents  a block 
diagram  of  the  audio/video  recording  capability  desired.  Two 
video  cameras  will  be  provided,  with  the  option  for  connecting  a 
special-purpose  camera  instead  of  either  of  the  two  standard 
cameras.  It  is  desired  to  record  complete  images  from  both 
video  cameras,  however,  in  lieu  of  this,  image  control  is  _ 
necessary  to  allow  combining  the  outputs  of  the  two  cameras  into 
a single  composite  picture.  Audio  recording  of  crew  communica- 
tions is  required,  but  the  upper  frequencies  of  the  available 
bandwidth  is  needed  for  recording  discrete  parameters  as  audio- 
encoded  signals.  The  audio/video  recorder  must  permit  remote 
control  in  synchronization  with  other  recording  devices. 


The  video  and  photo  information  to  be  recorded  in  each 
training  application  is  listed  in  Table  2 for  the  F-4  and  C-l 
aircraft;  other  aircraft  are  expected  to  require  similar 
application  of  video  and  photo  recording.  In  addition  to  the 
use  of  video/photo  recording  shown  in  Table  2 for  combat-crew 
training  performance  measurement,  the  use  of  a helmet-mounted 
camera,  instead  of  one  of  the  standard  video  cameras,  is 
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Figure  5.  Audio-Video  Recording  (AVR) 


TABLE  2 


VIDEO/PHOTO  DATA  FIELDS  BY 
TRAINING  PHASE 


PHASE 

\ 

VIDEO/PHOTO 

F-4 

C-141 

1A 

IB 

AUX 

1A 

IB 

AUX 

Transition 

PP 

W 

? 

PP 

W 

? 

Ins truments 

PP 

W 

? 

PP 

CC 

• 

Formation 

PP 

W 

PP 

w 

Intercept 

PP 

SC 

- 

- 

Air  Refueling 

PP 

W 

Air  Drop 

- 

- 

PP 

w 

T 

Ground  Attack 

PP 

GC 

- 

- 

- 

Dart  Firing 

. PP 

GC 

- 

- 

- 

Air  Combat 

PP 

GC 

- 

- 

- 

LEGEND:  PP:  Pilot1 s Panel 

W:  Windscreen 

SC:  Scope  Camera 

GC:  Gun  Camera 

CC:  Center  Console’ 

T:  Terrain  Directly  Below 

?:  Possible  Use 
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contemplated  to  acquire  information  on  targets  the  crew  should 
keep  under  surveillance,  and  behavioral  measurement  of  the 
crewmember ' s eye  fixation  patterns. 

1.  Audio/ video  recorder.  Specifications  for  an  ^io/vi^eo 
to^^fLSre^s^ti^^evfdenfupofptaiLck  for  dSta  Processing 

teiM?  sr  ssrssif  -f 

be  available  in  spite  of  the  severe  environments  encountered^  ^ 

during  combat-crew  training.  Further , ^ the^sY^tgm  must 

use  and  maintain.  The  design  and  of  the 

consider  the  limitations  imposed  by  cockpit  operatio  ' -~~ 

crew,  and  must  not  impose  any  dangers  to  personne  Y* 

Off-the-shelf  video  recording  systems  have  been  previously 

tested  in  combat-crew  training  environments . The  results 
specific  tests  of  interest  are  available  m the  following 

documents . 

Department  of  the  Air  Force,  Headquarters,  uf^  tactical 

Fighter  Weapons  Center.  TAC  Test  69  4£, — 

Re?orfing_sVstem _JAVRS)_.  "Nellis  AFB , Nevada,  id  bep 

l57o^ 

nonsrfmpnt  of  the  Air  Force,  Headquarters,  USAF  Tactical 
D€PFighter  Seapons  Center.  A-7D  Airborne  video  Record^ 
System  (AVRS) . TAC-TR-70A-113F , Nellis  AFB,  Nevada, 

February  1971. 

Fitzgerald,  J.A.,  and  Moulton,  D.L.  Evaluation. of  airborne 
Audio-Video  Recording  as  a Tool  for  Training  m _ - — 

Metical  Fighter.  AFHRL  (?T) -TRM-1  / , Air  Force 
Command,  Brooks  Air  Force  Base,  Texas,  October  1 

A number  of  unsatisfactory  incidents  are  rep ^J^tro- 

documents,  including  ioss  of  rnfor^tion  due  to  glare, 

magnetic  interference,  vrbratron,  , ”^^^allltiin  of  the  audio/ 

video€recording°system  muft  eliminate  these  problems  so  that  data 
collection  i^not  impaired  during  system  test  and  operatron. 


♦Devices  are  also  available  for  this  purpose  which  allow  direct 
digital  recording  of  eye  movements. 


taw* 
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TABLE  3 


AUDIO-VIDEO  RECORDING  AND  PLAYBACK  SPECIFICATIONS* 


STANDARD  AMERICAN  TV 

VIDEO  QUALITY;  Horiz.  and  vert,  resolution,  500 

lines  desired,  400  lines  minimum. 

700-800%  contrast,  8-10  shades  of  gray. 

Greater  than  40  DB  signal- to-noise  ratio. 

LENSES;  (1)  Zoom  type,  (2)  Fixed-outside 

viewing,  (3)  Fixed-outside  viewing,  very  wide  angle, 

(4)  Fixed-cockpit  viewing.  ' 


ILLUMINATION;  25-10,000  Foot-Candles;  automat 

sensitivity  control  over  this  range,  less  than  approx.  1- 
second  response  time,  rapid  recovery  from  direct  exposure 


to  sun. 

AUDIO;  Frequency  response  100-10,000 

— _jmin. ) HfS ; input  from  aircraft  interphone  syste,  earphone 
{T  speaker  output. 


INTERFERENCE;  ' ' (Audio  & Video)  Low  noise  back- 

ground  for  audio  comrfiTinic^tion  recording  (filters  for 
aircraft-induced  noise  required-!.,  negligible  interference 
with  either  video  or  audio  signals  due  to  aircraft  radio 
transmission,  avionics,  or  weapons  firing."  — . 

CONTROLS,  LIGHTS,  DISPLAYS;  Operation  with  either  hand  in 

aircraft  cockpit  with  full  flight  suit;  lights  adjustable 
for  day  and  night  operation;  no  restriction  to  crev7-^ 
visibility,  mobility,  or  safety;  amount  of  tape  remaining 
indicator  desired. 

SIZE,  WEIGHT,  POWER,  COOLING;  Suitable  for  fighter  aircraft 
installation;  without  significant  cockpit  modification, 
restriction  of  visibility,  mobility,  or  safety;  installa- 
tion clear  of  crew  ejection  envelopes. 

ENVIRONMENT;  Dictated  by  normal  combat-crew 

training  (e.g.,  altitude  to  50,000  ft.  MSL,  weapon  firing, 
turbulence,  maneuvering  ± 8G) 

MAINTAINABILITY;  Convenient  access  for  maintenance 

and  tape  handling;  provision  for  ground  operation  without 
aircraft  power;  set-up  and  test  equipment  to  be  included 
in  system,  including  operations  and  maintenance  documenta- 
tion. 


RECORDING  TIME; 


30-minutes  minimum. 


REMOTE  CONTROL;  On/off,  start/stop  controlled 

remotely  in  unison  with  other  recording  devices. 


*The  difficulties  encountered  in  previous  tests  must  be  removed;  i.e., 
specifically  those  in  (1)  TAC  Test  69-4F,  (2)  TAC-TR-70A-113F, 

and  (3)  AFHRL(FT) -TRM-17. 
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2.  Image  control . Simultaneous  synchronized  recording  of 
two  complete  pictures  from  two  video  cameras  is  desired.  If 
two-channel  video  recording  is  not  possible,  then  extensive  image 
controls  are  necessary  to  attempt  to  approximate  the  desired 
information  and  resolution  by  allowing  the  available  image  area 
to  present  the  most  important  information  from  each  camera. 
Methods  desired  for  image  control  are  listed  in  Table  4;  the 
design  should  include  as  many  of  these  features  as  possible,  as 
well  as  others  which  may  become  apparent  to  enhance  data 
collection  ability. 


* 


3.  Discrete  encoder.  Recording  of  discrete  events  is 
necessary  to  measure  quantities  which  infrequently  change,  and 
quantities  which  are  important  to  measurement  control  (i.e., 
start-stop  logic) . Unfortunately,  many  of  these  (e.g. , weight 
off  the  wheels,  flap  settings)  are  either  invisible  or  out  of 
the  field-of-view  of  video/photo  cameras.  Incorporation  of 
these  quantities  can  greatly  increase  the  value  of  video  record- 
ing; consequently,  a method  for  discrete  recording  should  be 
developed  if  practical.  A method  is  suggested  here  for  encoding 
discrete  information  as  audio  tones  within  the  available  audio 
recording  bandwidth,  however,  other  approaches  may  be  available 
for  consideration.  As  depicted  in  Figure  6,  communication 
recording  should  occupy  the  frequency  spectrum  300-3000  Hertz, 
leaving  the  higher  frequencies  available  for  recording  a tone  for 
each  binary  parameter.  The  number  of  parameters  which  may  be 
recorded  in  this  manner  depends  on  the  audio  bandwidth  actually 
available,  the  bandpass  of  audio  filters  during  playback,  and  the 
stability  of  tone  oscillators  for  recording.  A minimum  of  10 
discrete  recording  channels  should  be  provided,  but  24  or  more 
channels  could  be  useful. 

f 

A selectable  time  bit  (at  1,  2 or  4 second  intervals) , 
derived  from  the  time  code  generator,  must  be  recorded  as  one  of 
the  discrete  channels  to  aid  processing  during  playback. 

4.  Set-up  and  test  equipment.  The  audio/video  recording 
system  will  also  include  equipment  for  check-out  and  test  of  the 
video  system  (other  than  standard  test  equipment) , and  for  set-up 
of  the  specific  irftages  required  for  recording. 

Audio/digital  recorder  (ADR) . The  parameters  allocated  to 
audio7digital  recording  are  listed  in  Table  5 for  key  training 
phases.  These  parameters  can  be  divided  into  the  following 
categories:  (1)  aircraft  parameters,  (2)  control  parameters, 

(3)  binary  discrete,  parameters f (4)  time,  and  (5)  communications. 

A minimum  of  24  discrete  recording  channels  are  needed  in 
addition  to  those  required  for  digital  time  code  recording.  Of 
course,  an  audio  recording  channel  is  necessary*  Otherwise  there 
are  16  channels  which  require  analog- to-digital  conversion;  these 
normally  must  be  sampled  twice  a second,  but  a portion  of  these 
channels  must  be  sampled  more  often  to  allow  spectral  analyses 
and  precise  training  as  shown  in  Table  6. 


TABLE  4 


IMAGE  CONTROL  METHODS 


1. 

Horizontal  Split 

X% 

Camera 

A, 

Y% 

Camera  B. 

2. 

Vertical  Split 

X% 

Camera 

A, 

Y% 

Camera  B. 

3.  Inset  Camera  A picture  within  Camera  B picture. 

4.  Alternate  Camera  A,  Camera  B at  .1,  .5,  1,  2,  3, 

5,  10  second  and  1,  2,  5 minute  intervals 
(or  continuous  adjustment  of  interval) . 

5.  Sequence:  (1)  split  screen,  (2)  full  image  of 

Camera  A on  Event  A,  (3)  return  to  split 
screen  on  Event  B.  (Event  A & B are  aircraft 
discrete  parameters  designated  for  digital 
recording.) 

6 . Pan  camera  back  and  forth  within  the  cockpit  at 

selectable  rate. 


DISCRETE  PARAMETER  RECORDING* 


COMMUNICATION 

RECORDING 


123456789  10  11  12 


v 


3000 

HERTZ 


10,000 


<J — AVAILABLE  AUDIO 

BANDWIDTH 


♦Minimum  of  10  channels, 
24  or  more  desired. 


Figure  6.  Discrete  Audio  Recording. 
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TABLE  5 

AUDIO/DIGITAL  RECORDER  PARAMETERS 


>H 

u 


o 

u 

u 

< 


AIRCRAFT  PARAMETERS 

1.  Pitch  (Pitch  Rate) 

2.  Roll 

3 . Heading 

4.  Airspeed 

5.  MACH 

6.  Altitude 

7.  Vert.  Vel. 

8.  Angle  of  Attack 

9.  Acceleration  (G) 

10.  Power  (RPM,  EPR,  TIT, 

Fuel  Flow) 

11.  Fuel  Quantity 

CONTROL  PARAMETERS 

1.  Stick  (Pitch) 

2.  Stick  (Roll) 

3.  Rudder 

4.  Flap  Position 

5.  Stab  Trim  Position 

BINARY  DISCRETE  PARAMETERS 

1.  Thrust  Reverse 

2 . Speed  Brakes 

3,4.  Main,  Nose  Gear  Contact 

5.  Nose  Steer  Engaged 

6.  Gear  Select 

7.  Drag  Chute 

8.  Wheel  Brakes 
9,10.  Red,  Green  Light 

11.  Weapon  Release  (Pickle) 
12,16.  Crewmember  Voice  Switch 
17,19.  Marker  Beacon 
20,24.  Event  Marker 


X 

X 

X 

X 

X 

X 

X 

±1  degree 

X 

X 

X 

X 

X 

X 

X 

±1  degree 

X 

X 

X 

X 

X 

X 

X 

X 

±1  degree 

X 

X 

X 

X 

X 

X 

X 

X 

+1  knot 

X 

X 

X 

+.02  MACH 

X 

X 

X 

X 

X 

X 

X 

X 

±10  feet 

X 

X 

X 

X 

X 

X 

X 

X 

+50  fpm 

X 

X 

X 

X 

X 

±1  unit 

X 

X 

X 

X 

+ .5G 

X 

X 

X 

X 

X 

X 

X 

X 

+1%  Full 

Scale 

X 

X 

X 

+5%  Full 

Scale 

X 

X 

X 

±5%  Full 

Scale 

X 

X 

X 

±5%  Full 

Scale 

X 

±5%  Full 

Scale 

X 

X 

X 

±5%  Full 

Scale 

X 

X 

X 

+5%  Full 

Scale 

X 

X X 

X 

X 

— — 

X 

— — 

X 

X 

— — 

X 

—— 

X 

— — 

X 

— 

X 

XXX 

— 

XXXXXXXXX  — 
X 

XXXXXXXXX  — 


TIME 

1.  GMT  (Range  Time) 


XXXXXXXXX  Hrs , Min,  Sec, 

1/100  Sec. 


COMMUNICATIONS 


XXXXXXXXX 


TABLE  6 


DIGITAL  RECORDING  CHANNELS  AND  SAMPLING  RATE* 


16 

10 

5 


2 

10 

20 


*In  addition  to  Discrete  Parameters  and  Time. 


rras  - 

maintain  an  airspeed  of  135-5  k ' . h pilot  may  be  scored 

ssssffi.  SCTSs/g  4rsM  ^10% 

permitted)  . 

A gross  block  diagram  o£  the  Audio/Digital  ^order 

presented  in  Figure  7;  the  principa  pa  -s  ar  d As  there 

^ra^raU^d^g"^ 

<>ui<Jance- 

._AEr.:  ^^;”“^iH:SirS%rsi.s 

permit  digital  recorder  chec  ° same  equipment  should 

prior  to  each  data  collection  flight  the  same  equiP^  minimal 

facilitate  diagnosis  and  repair.  Maint. B*a  c collection  takes 

intense  schedule),  is  extremely 

important  to  the  intended  applications. 

anv-marv  camera  (AC).  An  auxiliary  camera  is  needed^as  a 

:ointsdUarenLptcdtedPU  tenllZltryV  External  controls  will 
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TABLE  7 


AUDIO/DIGITAL  RECORDER  SPECIFICATIONS 


parameteRS/ACCURACY,  ~ As  indicated  in  Table  5 (minimum, 

growth  capacity  desired. 


INTERFERENCE: 


Controlled  to  achieve  desired  accuracy. 


parameter  SAMPLING  HATES!  Twice  per  second <16  + discretes  + 

, 10  timeF per  second  (10  + discretes 

times  per  second  (5  + discretes  + time). 


,Mln> 

“eate?  output;  negligible  background  norse,  low 
distortion. 


T IME  CODE  RECORDING: 


(Hrs,  Min,  Sec,  1/100  Sec.) 


Through  conversion  equipment  interface 
OUTPUT  • “ 

aeneral  purpose  digital  computer. 

Audio  oS?pu?  and  digital  display  of  digital  time  and 

discrete  parameters  during  playback. 


REMOTE  CONTROL: 


other  recording  devices. 


On/Off,  Start-Stop,  together  with 


RECORDING  TIME: 


30-minutes  minimum. 


SSS. blffir  day'and 

night  operation;  no  restriction  to  crew  visibili  y, 
mobility,  or  safety. 


ejection  envelope) . 


pmvtbokmfnt*  Environment  dictated  by  normal  combat- 

ENVIRONMENT.  _ _ altitude  to  50,000  ft.  MSL,  weapon 

crew  training  (e.g.,  aitituae  t-u  au, 

firing,  maneuvering  ±8G) . 


matmtrtnabilitY*  Convenient  access  for  maintenance, 

and  tape  handling,  Provision  for  ground 
operation  without  aircraft  power;  tesu  stand . to 
provided  (other  than  standard  test  equipment)  ; minimum 
maintenance  personnel  requirements. 


■ 


expose  the  film  in  either  a motion  picture  sequence,  or  as 
individual  frames  (from  0.1-  to  10-second  intervals). 

Interphone  Interface  (II).  The  interphone  interface  (see 
Figure  8)  is  to  permit  selection  of  the  communications  from  any 
crewmember,  and  to  generate  a discrete  signal  identifying  which 
crewmember  is  speaking.  Since  the  available  interphone  system 
may  be  switched  so  that  all  crewmembers  of  interest  are  not 
recorded,  an  audio  mixer  is  specified  to  combine  the  outputs 
desired  for  both  AVR  and  ADR  audio  recording.  Voice-operated 
switches  installed  at  each  microphone  are  suggested  to  generate 
a discrete  signal  whenever  each  crewmember  speaks. 

Recording  Controls/Displays  (RCD) . Recording  controls  and 
displays  provide  manual  and  programmed  control  of  recorder 
functions,  and  information  for  the  human  operators  using  the 
system  during  set-up  and  data  collection.  A block  diagram  of  the 
RCD  is  presented  in  Figure  9a. 

A time-code  generator  and  event-mark  controls  are  included 
in  the  RCD  to  be  recorded  on  discrete  recording  channels.  These 
discrete  signals  are  also  used  to  turn  the  recording  system  on 
and  off,  or  initiate  timers  which  will  switch  the  system  on  or 
off  after  a specified  interval.  As  indicated  in  Figure  9b,  the 
following  sequence  is  desired:  (1)  recording  is  initiated  by 
either  event  A (one  of  the  discrete  signals  recorded)  or  manually, 
(2)  recording  is  stopped  manually,  at  event  B,  or  after  a 
specified  time,  (3)  recording  is  initiated  manually,  at  event  C, 
or  after  a specified  time,  and  (4)  recording  is  stopped  again 
manually,  at  event  D,  or  after  a specified  time.  For  current 
purposes  an  event  for  recording  control  may  be  a manual  input 
(an  event  mark),  an  aircraft  discrete  parameter,  or  a specified 
time  bit  from  the  time-code  generator. 

It  is  recommended  that  the  RCD  be  divided  into  two  units 
with  controls  and  displays  as  shown  in  Table.  8.  Unit  1 provides 
information  and  control  useful  to  a member  of  the  flight  crew 
during  training;  while  it  should  be  small  and  non-obtrusive , 
operation  must  be  possible  in  a cockpit  environment  with  full 
flight  dress.  Unit  2 permits  set-up  of  recording  programming 
prior  to  a planned  mission;  it  may  be  normally  utilized  by  a 
ground  technician.  However,  some  aircraft  and  circumstances  may 
dictate  the  presence  of  an  airborne  experimenter;  in  this  case, 
both  units  must  be  cc -located  at  his  station. 

Personnel  tasks  during  recording.  Gross  personnel  tasks 
for  recording  of  aircraft/simulator  data  are  listed  in  Table  9. 
Further  task  definition  will  be  possible  when  detailed  equipment 
designs  are  selected. 

Field  Data  Collection  Station  (FDCS) 

The  Field  Data  Collection  Station  (FDCS)  will  provide  data 
collection  at  remote  places  such  as  at  the  runway,  the  bombing 
range,  or  ground  surveillance  radar.  For  these  purposes, 
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♦Voice-Operated 

Switch 


Discrete 

Recording 


Figure 


8.  Interphone  Interface. 


AVR 
Discrete 
Encoder 


a.  BLOCK  DIAGRAM 


RECORDER 

ON— 



RECORDER 

OFF — 

ft 

A 

\ 

B 

< 

C 

k 

Time 

Event  A 

Event  B 

Event  C 

Note  1; 

t i / ta 

Programmable 

Note  2: 

Events  (A,B,C) 

Can  be:  (1)  Manual 

Input 

(2)  Acrft  Discretes 

(3)  Specified  Time 

b.  PROGRAMMED  RECORDING  SEQUENCES 


Figure  9.  Recording  Controls  and  Displays 
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TABLE  8 

RECORDING  CONTROLS/DISPLAYS 


UNIT  1 (For  Flight  Crewmember  or  Airborne  Experimenter) 


On-Off,  Start-Stop  Controls  for  Aux.  Camera,  AVR  and  ADR 
Event  Marker  Controls 
Digital  Time  Display 
Time  Setting  Controls 

Tape  Remaining  Display  for  AVR  and  ADR 


UNIT  2 (For  Ground  Technician  or  Airborne  Experimenter) 


Record  Programming: 

On-Event/Manual  Start/Interval  A 
Interval  B/Off-Event 
Interval  C/On-Event 
Interval  D/Off-Event 


Video  Framing: 

Horiz.  Split  - % Camera  A/B 
Vertical  Split  - % Camera  A/B 
Inset  - % Camera  A/B 

Time  Split  - 0.1,  .5,  1,  2,  3,  5,  10  seconds 
Event  A Full  Camera  A,  Return  to  Split  on  Event  B 


Aux.  Camera: 

Exposure  Rate:  Motion,  .1,  .5,  1,  2,  3,  5,  10  seconds 


i 

1 
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Test,  Replace/Repair,  Preventative  Maintenance,  Adjust 
LOAD  TAPES 

i 

PERFORM  CALIBRATION  AND  PRE-FLIGHT  RECORDING 
SET-UP  FOR  DATA  COLLECTION 

Record  Programming:  On/Off  Events  in  Time  Intervals 

Video  Framing:  Split,  Inset,  Time,  Sequence 

E 

Aux.  Camera  Exposure 
INFLI GHT/SIMULATOR 
System  On 

Manual  Start/Stop  as  Required 
Manual  Event  Mark  as  Appropriate 
System  Off 

PERFORM  POST-FLIGHT  RECORDING 


UNLOAD  TAPES 


portable  equipment  with  self-contained  power  sources  is  needed. 

A simple  data  collection  station  is  desired;  Figure  10  depicts 
the  major  equipment  items,  a camera  with  timing  control,  an  auaio 
tape  recorder,  and  transceivers. 

Camera.  Camera  specifications  for  the  FDCS  are  much  the 
same  as  for  the  Auxiliary  Camera  for  aircraft/simulator  data 
collection.  Therefore  it  would  be  desirable  for  these  camera 
systems  to  be  compatible.  An  accurate  clock  is  needed  in  the 
corner  of  each  frame  of  the  FDCS  camera  to  allow  correlation  with 
data  collected  elsewhere.  The  exposure- timing  mechanism  require- 
ments are  the  same  as  for  the  auxiliary  camera  (motion,  .1,  .5,  1, 
2,  3,  5,  10  seconds). 

Audio  tape  recorder.  A small  hand— carried  cassette— type 
tape  recorder  is  required  for  recording1  of  ground  station 
communications  and  other  verbal  information. 


Transceivers.  Transceivers  will  be  used  for  data  collection 
team  c ommun i ca t i o n s between  the  FDCS  crew  and  the  remainder  of 
the  data  collection  team  at  the  data  playback  and  processing 
facility,  and  to  monitor  flight  crew  transmissions  during  training 
missions.  The  frequency  and  power  of  these  units  is,  of  course, 
governed  by  Air  Force  regulations,  but  maximum  communication 
capability  within  these  constraints  is  desired  (these  units  may 
be  provided  from  the  Air  Force  Inventory) . 


Much  of  the  information  presented  during  mission  briefings 
relates  to  expected  performance  and  appropriate  performance 
measurement;  information  produced  during  debriefing  can  modify 
the  briefed  data,  and  can  provide  a source  of  subjective  measure- 
ment for  correlation  with  measured  performance  indicators.  Many 
of  these  briefings  are  so  rich  in  information  that  a small  hand- 
carried  cassette— type  recorder  may  be  necessary  for  recording  for 
later  transcription.  A small  camera  may  also  be  useful,  if 
detailed  briefing  displays  are  used  and  copies  of  briefing  slides 
are  unobtainable. 


Documentary  Data  Collection  Station  (DDCS) 


Desired  performance  and  measurement  control  parameters  will 
also  be  obtained  from  documents  such  as  the  Dash-One  Technical 
Order,  Phase  Manuals,  and  operational  publications.  No  special 
equipment  is  believed  necessary  for  this  form  of  data  collection, 
although  digital  encoding  equipment  for  computer  processing 
(e.g.,  teletype)  is  necessary  but  will  be  discussed  in  regard  to 
the  Data  Playback  Station. 
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TIMING 

CONROLS 
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AUDIO  TAPE 
RECORDER 


Figure  10.  Field  Data  Collection  Station 


External  Data  Collection  Station  (EDCS) 


Data  will  also  be  produced  on  occasion  by  an  external 
source  which  must  be  correlated  with  data  collected  during  combat- 
crew  training.  For  example,  external  data  may  be  produced  by 
another  experimental  investigation  performed  elsewhere,  or  the 
output  of  a ground  multiple-target  tracking  radar  on  an  instru- 
mented test  range.  These  data  must  be  provided  to  the  current 
facility  in  a form  permitting  processing  with  the  available 
general  purpose  digital  computer.  Under  this  assumption,  no 
special  data  acquisition  need  be  provided  for  externally 
produced  data. 

Data  Playback  Station  (DPS) 

The  Data  Playback  Station  (DPS),  as  shown  in’Figure  11,  must 
permit  the  transformation  of  data  collected  through  the  five 
avenues  shown  in  Figure  2 into  a digital  format  appropriate  to  the 
general  purpose  computer.  The  digital  magnetic  tape  requires 
only  routine  human  operator  activities  to  load  the  data  into 
computer  storage,  although  the  audio  recording  portion  is 
necessarily  a manual  processing  task.  All  other  types  of 
information  require  manual  processing,  and  eventually,  typing  of 
formatted  data  onto  a punched  paper  tape. 

Video  data  processing.  Video  information  will  be  reproduced 
through  two  video  monitors  if  dual-channel  recording  is  possible, 
otherwise,  one  will  be  used  if  split-screen  merging  of  pictures 
is  performed.  Communications  and  the  special  discrete  audio 
channels  are  reproduced  simultaneously  with  the  video  information, 
thus  the  operator  can  ascertain  who  is  talking,  and  the  status  of 
gear,  speed  brakes,  etc.  (if  these  are  chosen  for  recording) 
while  analyzing  the  video  content. 

The  discrete  information  is  also  useful  to  aid  searching  for 
key  events  to  initiate  measurement  activity  sequences.  Automatic 
measurement  processing  keys  on  clearly  identifiable  events 
whenever  possible  (i.e.,  discrete  events).  Unfortunately,  these 
events  often  do  not  appear  in  the  video  picture.  The  human 
operator  would  then  have  to  laboriously  search  the.  video  tapes  to 
infer  from  the  cockpit  instruments  that  an  event  has  occurred 
which  is  important  to  measurement.  The  special  discrete  channels 
then  provides  the  human  operator  with  the  advantage  of  the  same 
information  provided  an  automatic  processor.  Since  it  is 
normally  desired  that  the  situation  at  or  near  each  event  be 
analyzed,  playback  control  features  are  required  to  stop  the 
video  tape  at  events  occurring  on  specified  discrete  channels. 

(For  example,  set  playback  controls  so  that  the  tape  will  stop 
when  weight  is  off  the  wheels,  allowing  conditions  at  lift-off 
to  be  recorded  for  computer  entry.) 

To  further  aid  the  human  operator  in  sampling  performance  at 
constant  time  intervals,  or  just  to  advance  the  tape  a known 
amount,  it  is  required  that  the  tape  be  advanced  on  command  for  a 
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specified  number  of  seconds.  -It  was  earlier.,  specified  that  a 
time  bit  (1,  2,  or  4 seconds)  be  recorded  on  one  discrete 
channel  so  that  these  bits  can  be  counted  on  playback  for  time 
control  (i.e.,  the  control  requested  is  to  advance  N events  on 
the  time-bit  discrete  channel) . 

A TELETYPE1,  or  similar  device,  will  be  used  to  punch  paper 
tape  to  enter  video  data  into  computer  storage. 

Digital  data  entry.  Digital  data  are  expected  to  come  from 
either  external  sources  or  the  audio/digital  recorder  system. 

External  digital  data  should  be  acceptable  in  magnetic  tape, 
punched  card,  or  paper-tape  form;  any  of  these  are  directly 
read  by  the  general  purpose  computer,  although  a magnetic  tape 
copy  may  be  made  for  high-speed  re-reading. 

Audio/digital  recording  can  result  in  two  acceptable  forms: 
a digital  magnetic  tape  which  can  be  converted  into  a magnetic 
tape  readable  on  standard  computer  magnetic  tape  units,  and  a 
digital  magnetic  tape  which  is  read  through  special  conversion 
equipment  directly  through  an  electrical  interface  into  the 
general  purpose  computer.  The  most  cost-effective  of  these 
alternatives  should  be  used.  Neither  method  requires  any 
significant  manual  processing  activities  to  enter  data  into  the 
computer  data  base. 

Audio  data  processing.  Audio  data  processing  is  necessarily 
a manual  task.  The  difference  between  audio  data  processing  with 
audio/video  playback  and  the  audio/digital  playback  is  the  method 
of  correlating  the  audio  information  with  other  performance 
parameters.  With  audio/video  playback,  the  magnetic  tape  must  be 
stopped  at  specific  auditory  events  (e.g.,  commands),  selected 
performance  parameter  values  noted,  and,  auditory  code  and 
values  punched  on  paper  tape.  With  audio/digital  playback,  the 
magnetic  tape  must  be  stopped  at  specific  auditory  events,  the 
time  noted  from  a display  of  the  digital  time  code,  and,  auditory 
code  and  time  punched  on  paper  tape  so  that  the  auditory  informa- 
tion can  be  correlated  with  appropriate  data  samples  read  from 
the  digital  magnetic  tape. 

Photo  data  processing.  Photo  data  processing  requires  many 
of  the  same  procedures  as  video  data  processing,  although  time- 
lapse  photography  is  likely  to  be  the  primary  mode,  and  photo 
analysis  must  proceed  without  benefit  of  the  special  discrete 
channels  used  with  video.  Information  may  be  located  by  (1) 
searching  for  conditions  on  the  cockpit  instruments  or  outside 
view,  and  (2)  finding  specific  frames  or  indications  on  the  panel 
clock  as  directed  by  information  from  the  audio  or  digital  — 
information.  The  resulting  information  must  be  coded  onto 
punched  paper  tape,  normally  including  time  values  to  allow 
correlation  with  data  from  other  sources. 


♦Trademark  of  the  Teletype  Corporation. 
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Manna],  data  processing  workstation.  Manual  processang  tasks 
identified  in  the  previous paragrapns  are  summarized  in  Taoie  1 . 
The  design  of  the  data  processing  workstation  is  important  to  t 
efficiency  and  accuracy  of  the  entire  training  measurement 
system;  its  design  should  carefully  consider  huruan  °£e^or 
requirements  and  limitations.  It  should  be  noted  that  this 
workstation  may  be  used  for  training  feedback  and  critique  by 
training  personnel?-**  well  as  used  for  data  processing  tasks. 

A minimum  list  of  manual  processing  equipment,  ^trols  and 
displays  is  presented  in  Table  11)  other  controls  and  displays 
mayPbeYadded  to  enhance  the  dual  purpose  of  this  workstation. 


TABLE  10 


PERSONNEL  TASKS  DURING  DATA  PLAYBACK 


VIDEO 

Load  tape,  play 

Read  calibration  values 

as  dictated  by  measurement  script: 

1.  Use  automatic  control  to  (1)  stop  at  event 

on  specific  discrete  channel,  and/or  (2) 
advance  N seconds  — i.e.,  count  time  bits  on 
discrete  channel  (Example:  Advance  to  "gear-up"). 

2.  Use  fast/slbw/single-frame  forward/reverse 

controls  to  search  in  the  neighborhood  of 
measurement  interest  (Example:  Search  for 

out-of-tolerance  condition) . 

3.  Use  teletype  to  punch  paper  tape  for  computer 

entry  (Example:  Parameter  code  number,  para- 

meter value,  time). 

DIGITAL 

Load  tape 

Play  into  conversion  equipment  to  (1)  transmit  data 

directly  to  disc  storage,  or,  (2)  produce  compatible 
magnetic  tape  to  play  on  computer  magnetic  tape 
units. 

AUDIO 

Load  tape,  play,  stop  at  auditory  information  desired 
Video:  Note  parameter  values 

Digital:  Note  time  from  digital  recorder  discrete  display 

Punch  paper  tape  (Example:  Code  number,  time) . 

PHOTO 

Load  film,  wind  film  through  viewer 
as  dictated  by  measurement  script: 

1.  Search  for  conditions  indicated  by  instruments , 

* and  read  specified  valued,  and/or, 

2.  Find  specific  time  indications  on  clock  (as 
derived  from  digital  time  readout)  or  a specific 
frame  number  and  read  specified  values. 

3.  Punch  paper  tape. 
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III.  DATA  PROCESSING  SUBSYSTEM 


The  system  to  process  training  research  measurement  as  it  has 
been  developed  in  this  study  is  described  in  teras  of  the  charac- 
teristics of  (1)  desired  hardware  and  (2)  associated  software. 

Data  Processing  Hardware 

It  is  desirable  to  have  all  measurement  processing  tools  in 
a central  location  ready  for  use  when  they  are  needed;  therefore, 
a dedicated  processing  facility  is  assumed  as  shown  in  Figure  12. 
Most  measurement  processing  can  be  performed  on  general-purpose 
computing  equipment;  however,  if  data  are  to  be  extracted  from 
airborne  instrumentation,  special  conversion  equipment  is  needed. 

Data  Processor.  The  heart  of  the  measurement  data  proc- 
essing  facility  is  a general  purpose  digital  computer.  Based 
on  experience  with  many  inflight  and  simulator  expermients  (see 
Table  12),  the  computer  should  have  about  16,000  to  32,000  words 
of  memory,  a word  size  of  at  least  16-bits  but  preferrable  24- 
bits  , and  a basic  operation  time  of  no  more  than  one  to  three 
microseconds.  Much  of  the  tuility  of  the  system  for  measurement 
and  data  analysis  depends  on  the  presence  of  the  following 
peripheral  equipment: 

(1)  Magnetic  Tape  Units.  At  least  two  magnetic  tape 
units  with  7 or  9 track  and  variable  density  capability  should 

be  provided  as  a means  to  store  program  and  data  files,  a vehicle 
for  entry  of  externally  collected  data  (possible  from  airborne 
instrumentation  recordings),  as  as  an  intermediate  output  medium. 
Of  course,  a magnetic  tape  controller  should  accompany  the  units. 

(2)  Disk  Units.  Although  it  is  possible  to  operate 
without  disk  storage  for  many  small  experimental  efforts,  the  use 
of  disk  storage  can  speed  operations,  increase  capacity,  provide 
efficient  program  and  data  file  storage  and  retrieval,  and  permits 
the  use  of  powerful  software  structures  and  operating  routines. 

A minimum  of  two  logical  disk  units  are  required;  four  logical 
units  are  preferred. 

(3)  Line  Printer/Plotter.  The  line  printer  permits 
high-volume  output  in  a timely  fashion,  which  is  necessary  for 
data  listings  and  multiple  statistical  analyses.  A minimum 
speed  of  300  lines  per  minute  is  desired.  SS,nce  almost  all 
performance  measurement  output  requires  plotting  for  inter- 
pretation, a plotting  capability  is  quite  desirable.  Recent 
hardware  development  suggest  that  a combination  printer/plotter 
may  provide  both  capabilities  at  a reasonable  cost. 

(4)  Teletype.  A teletype  or  a console  typewriter  is 
required  for  program  control,  system  monitoring  and  manual  data 
entry . 
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TABLE  12. 


DATA  PROCESSING  SYSTEM  HARDWARE 

Processor 

Memory  (16-32K  words) 

Word  Size  (18-24  bit) 

Cycle  Time  (1-3  microseconds) 

Floating  Point  Hardware 
Card  Reader 

Magnetic  Tape  Units  (2)  + Controller 
Disk  (204  logical  units) 

Plotter /Printer  (at  least  300  lines/  min) 
Teletype  or  Input  Typewriter 
Paper  Tape  Reader/Punch 
CRT  Display 

Airborne  Magnetic  Tape  Conversion 
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(5)  CRT.  Since  infromation  must  be  provided  for 
training  feedback,  manual  data  entry,  software  and  data  editing, 
an  electronic  display  (cathode  ray  tube)  is  recommended. 


(6)  Paper  Tape  Reader  and  Punch.  Paper  tape  provides 
a low-cost  input/output  capability,  compatibility  with  other 
computers,  and  is  the  primary  input  medium  for  manually  reduced 
data  in  the  processing  system.  It  is  a standard  feature  with 
most  computers  and  Teletypes. 


(7)  Card  Reader.  A card  reader  permits  convenient 
entry  of  data  collected  from  external  sources  such  as  subjective 
data,  paper-and-pencil  measurement  forms  and  data  from  other 
experiments.  Computer  programs  can  be  manipulated  in  card  form. 


Airborne  Tape  Conversion.  If  both  the  data  format  and 
physical  size  of  airborne  magnetic  recordings  are  the  same  as 
the  magnetic  tapes  normally  produced  by  the  general  purpose 
digital  computer,  then  only  the  computer  magnetic  tape  units 
will  be  needed  to  process  them.  However,  this  is  not  likely  to 
be  the  case;  conversion  equipment  that  will  read  the  airborne 
magnetic  tape  and  produce  computer  compatible  tapes  will  then 
be  required.  The  exact  nature  of  such  equipment  will  be  a 
function  of  the  configuration  and  format  of  the  airborne  instru- 
mentation system. 


(1)  Configuration.  Although  the  tape  width  may  be 
computer  compatible,  the  reel  size  may  be  different  requiring 
that  the  tape  be  rewound  onto  another  reel.  If  a tape  cassette 
is  used  (such  that  the  tape  cannot  be  physically  removed) , then 
the  tape  must  be  electronically  copied;  a playback  unit  will  be 
required. 


C2)  Format.  Current  airborne  recording  technology 
dictates  a recording  format  which  is  different  than  that  used  in 
a general  purpose  digital  computer  to  maintain  accuracy  in  spite 
of  noise  and  recorder  irregularities.  Pulse  code  modulation  (PCM) , 
pulse  duration  modulation  (PDM)  and  frequency  modulation  (FM) 
techniques  are  used.  Conversion  equipment  may  be  needed  for 
several  types  of  recording  if  compatibility  with  currently  instru- 
mented aircraft  is  desired. 


Software 


Data  processing  system  software  requirements  fall  into  two 
major  categories,  (1)  the  executive  monitor,  and  (2)  measurement 
processing.  The  monitor,  together  with  its  utility  programs  and 
operating  system  provide  the  fundamental  software  tools  which  are 
used  to  construct  and  execute  user  programs  and  to  manipulate 
data  files.  Measurement  processingprograms  form  the  logic  for 
generation,  control  and  analysis  o£  performance  measurement  data. 


.i,., . 


Executive  Monitor.  The  monitor  supervises  all  input/output 
(I/O)  operations  and  traps  errors.  All  programs  are  initiated 
from  the  monitor.  Manual  return  to  the  monitor  should  be  pos- 
sible from  any  user  program  at  any  time  during  processing.  All 
programs  should  return  to  the  monitor  upon  normal  completion. 

The  monitor  should  be  able  to  operate  in  a manual  or  batch 
processing  mode.  In  the  batch  processing  mode  it  receives  its 
commands  from  the  batch  input  device,  usually  a card  reader  or 
paper  tape  reader.  Batch  processing  reduces  operator  inter- 
vention and  allows  rapid,  semi-automatic  processing  of  sequential 
programs.  Manual  mode  monitor  communications  with  the  operator 
should  operate  through  the  console  keyboard,  a typewriter  or 
teletype. 

The  monitor  should  provide  a device  independent  operating 
system,  wherein  the  assignment  of  program  logical *1/0  devices  to 
the  actual  physical  devices  are  handled  by  the  monitor  and  can  be 
changed  at  runtime  by  a monitor  command.  Typically,  I/O  device 
handlers  and  buffers  are  brought-into  memory  by  such  monitor 
assignments.  I/O  handlers  should  allow  asynchronous  operation 
so  that  processing  can  continue  while  I/O  is  taking  place. 

The  handlers  should  provide  the  capability  to  operate  in  a 
named  file  or  non-file  priented  mode.  In  the  file  oriented  mode 
several  names  files  may  be  contained  on  a particular  tape  or  disk. 
During  file  oriented  operations,  a specific  names  file  is  opened 
(initiated)  for  reading  or  writing.  In  the  non-file  mode,  the 
whole  logical  device  is  treated  like  a magnetic  tape  in  terms  of 
read,  write,  rewind  and  backspace  operations.  It  is  desireable 
to  be  able  to  open  a names  file  for  both  reading  and  writing, 
but  this  feature  is  not  possible  to  open  a names  file  for  reading 
and  writing,  then  it  is  important  that  the  handlers  provide  both 
file  oriented  and  non-file  oriented  service. 

. . " ■'  ',r 

In  addition  to  its  own  operating  system  and  housekeeping 

routines,  the  monitor  should  call  at  least  seven  utility  programs 
through  a simple  keyboard  command  in  the  manual  mode,  or  paper 
tape  commands  Cor  cards)  in  the  batch  processing  mode.  These 
routines  are  shown  in  Figure  13,  and  are  briefly  described  in  the 
following  text  since  they  are  common  in  modern  software  structures. 

Cl)  Text  Editor.  The  editor  is  intended  for  on-line  Use 
with  Teletype  alone , or  with  a combination  keyboard  and  CRT 
display.  The  purpose  of  the  editor  is  to  create  or  change  Ccor- 
rect)  a file  of  Ca)  text,  (b)  user  program  instructions  Csuch  as 
FORTRAN  or  Machine  Language  source  programs,  or  Cc)  numbers  while 
the  operator  is  on-line. 

The  editor  should  allow  the  operator  to  open  and  close 
files  by  name,  and  to  change  the  names  of  existing  files.  To 
assist  with  data  input,  provisions  for  tabulating  Ci.e.:  moving 

the  "carriage"  to  pre-defined  positions)  and  adjusting  the  tab 
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positions  should  be  incorporated.  There  should  be  provision 
for  fetching  n-lines  of  input  data  from  another  peripheral 
device,  such  as  a paper  tape  reader.  Once  a file  is  opened, 
the  operator  either  inputs  text  or  data  in  the  input  mode, 
or  changes  the  content  of  the  file  in  the  edit  mode.  A con- 
venient method  of  switching  between  modes  at  any  time  in  the 
file  should  be  provided. 


In  the  input  mode,  the  operator  types  text,  instructions 
or  data  into  the  file,  one  line  at  a time.  If  the  input  mode  is 
being  used  to  insert  several  lines  into  an  existing  file,  the 
remaining  portion  of  the  file  should  be  undisturbed  by  the  newly 
inserted  text. 


In  the  editing  mode,  must  text  editors  start  at  the 
first  (or  top)  line  of  a file,  and  bring  successive  or  requested 
lines  to  the  working  area  on  command.  Typically,  a line  is 
brought  to  the  working  area  by  (")  specifying  a line  number, 

(2)  asking  for  the  next  line  or  n-iines,  (3)  searching  the  file 
for  a statement  number  that  may  be  contained  at  the  beginning  of 
the  line,  or  C4)  searching  the  file  for  a character  string  that 
may  be  contained  on  the  desired  line.  Once  in  the  working  area, 
the  line  can  be  re-typed,  added  onto,  deleted,  or  any  defined 
character  string  within  the  line  can  be  changed.  A summary  of 
these  desired  editor  control  functions  is  shown  in  Table  13. 


Good  human  engineering  of  the  text  editor  control 
functions  is  essential  to  keep  the  operator  error  rates  and 
subsequent  machine  time  utilization  to  an  acceptable  level. 

Editor  commands  should  be  in  natural  language  where  possible, 
and  abbreviated.  For  example,  to  locate  the  variable  ISUM  in 
a FORTRAN  source  file,  a simple  locate  command,  "L  ISUM"  should 
e sufficient  to  search  the  file  for  ISUM,  and  to  bring  the  first 
line  containing  ISUM  to  the  working  area  fir  operation.  Whether 
or  not  the  line  containing  ISUM  is  echoed  on  the  output  device 
when  it  is  found  should  be  a function  of  a "verify"  switch, 
which  is  normally  on,  but  can  be  turned-off ‘with  a simple  com- 
mand such  as  "V  OFF." 


frequently  necessary  to  majcs  the  same  changes  to 
repetative  character  strings  throughout  a file.  One  command 
string  should  be  sufficient  to  do  so.  For  example,  a command 
such  as,  "L  C /ISUM/SUM"  (where  slashes  are  field  delimiters  for 
example  purposes  only)  should  cause  the  editor  to  search  the 
entire  file,  and  to  change  earch  ISUM  it  finds  to  SUM. 


A requirement  for  the  operator  to  completely  spell 
commands  (such  as  "LOCATE")  is  an  example  of  an  undesireable 
design  feature.  Similarily.  some  text  editors  allow„movement 
tnrough  the  file  in  a forward  direction  only,  which  can  lead  to 
error  and  consume  excessive  machine  time;  a simple  backup  com- 
mand should  be  provided.  Some  editors  tend  to  make  the  use  of 
line  numbers  mandatory  in  the  edit  mode;  although  it  may  be 
convenient  at  certain  times  to  use  line  number  references,  it 
should  not  be  mandatory  for  the  operator  to  do  so. 
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TABLE  13 

TEXT  EDITOR  CONTROL  FUNCTIONS 
% 

OPEN,  CLOSE  and  RENAME  a file. 

TAB  change. 

GET  n-input  lines  from  peripheral  device. 

FIND  a statement  number. 

LOCATE  a character  string . 

CHANGE  a character  string  within  a line. 

LOCATE  AND  CHANGE  repetative  character  strings  throughout 
the  file. 

RETYPE  a line. 

ADD  to  the  end  of  a current  line. 

OVERLAY  n-lines  with  an  equal  number  of  lines  to  be  input. 
DELETE  the  current  line  or  n-lines. 

PRINT  (output)  the  existing  line  or  following  n-lines. 
Bring  NEXT  line  or  n-lines  to  the  working  area. 

BACKUP  one  or  n-lines. 

NOTE:  Text  editor  should  be  called  from  monitor  with  single 

command . 


. ^ • -i  ■ , ^ File  Manipulator . The  file  manipulator  is  a 

?r°gr^n  whlch  can  be  called  from  the  monitor  by  a simple 
command  in  order  to  perform  the  file  operations  shown  in  Table  14 

SSSS  b“2!:1to%9UagJ  inPUt  t0  the  "eyb°ard'  ^ operator16  U* 

verifv  Lh  i ^transfer,  rename,  segment,  combine,  delete, 
verify,  and  copy  files  from  device  to  device  (or  within  the  same 
device  to  create  a new  file  with  a different  name  but  using  the 
S”!  the  °ld  fiie) . Although  file  manipulation  Is  9 
primarily  intended  for  named  file  oriented  devices,  it  is 
desirable  to  permit  a binary  block  transfer  or  copy  of  hon- 
i^.e  oriented  data.  This  would  provide  capability  to  dump  the 
contents  of  any  tape  (that  can  be  read)  onto  the  line  plinth 
for  examination,  or  to  copy  any  tape  that  can  be  read. 

„ . . (d)  FORTRAN  IV  Compiler.  FORTRAN  IV  should  provide 

a compiler- leveTTahguage  with  sufficient  power  for  thePgener- 

specific  additio^0rm^Ce  meas“reinent  user  programs,  with  several 
°ns*  computer  should  allow  (a)  a minimum  of 

frl  fnrm  +.  2nal  ^rraY  manipulation,  (b)  negative  DO  loop  indexing, 

d llT ASC  Wit!;  Character  field^  separated  by  commas?9 

e tnl  ch?racters  allowed  in  Hollerith  fields, 

rn  * thmetic  expression  as  a legal  array  subscript,  and 
(f)  any  array  element  as  a legal  subscript.  A single  command  to 
the  monitor  should  load  the  FORTRAN  compiler. 

.hn„1H  . When  the  compiler  is  loaded,  a single  command  string 
*d  be  suff^c^ent  to  identify  the  source  program  to  compill 

?hc  "s?aSd"  set°UtPU£  °Pt^°?S;-  0UtpUt  °PtionE  should  include 

listing  tel  soulre  w J-3S  bmary  object  program,  (b)  object 
isting,  (c)  source  listing,  (d)  error  only  listing  and  (e)  svmbol 

able  fo^sourr  ^ aYailable  in  some  systems,  it  would  be  desir- 
stirtfnJ  hS  lutings  from  the  compiler  to  show  the  relative 
rnr  b 9 address  °f  the  blnarY  (object)  instruction  set  that 

cha^r?^r?L^  each  source  program  instruction  line.  The  FORTRAN 
cnaractenstics  desired  are  summarized  in  Table  15. 

. (4)  Machine  Language  Assembler.  A machine  language 

programming  capability  should  be  provided  for  the  generation  of 

and^r^f115  ^ P0IT*AN  callable  subroutines  fo/processing 
Pu^P°se  I/O  operations  that  are  more  efficiently9 
handled  in  machine  language.  As  with  the  FORTRAN  compiler,  the 
fan9uage  assembler  should  be  callable  from  the  monitor 
th^cc^hi16  co^aneJ.  Once  loaded,  a single  command  string  to 
Jnd  a®se^bl®r  should  be  sufficient  to  identify  the  source  file 
and  standard  output  options  such  as  error  diagnoses,  symbol 
maps  and  cross-references. 

, , ,(5i..  ^def : Most  software  structures  require  a loader, 

execStiondS  ^oarY  °^ect  Pr°9rams  into  memory,  and  starts  their 
execution.  Two  versions  of  the  loader  are  required,  a load-and- 

go  version  which  loads  the  programs  and  initiates  execution,  and 
oad  and  wait  version  which  requires  an  operator  command  to 
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TABLE  14 


FILE  MANIPULATOR  CONTROL  FUNCTIONS 

TRANSFER  named  file  from  one  device  or  media  to  another. 
RENAME  a file  on  any  device  or  media. 

SEGMENT  a file  into  several  new  files. 

B RING-TOGETHER  (combine)  several  files  into  one  file. 
DELETE  a file. 

i 

VERIFY  the  existence  of  a file  on  any  device  or  media. 

COPY  the  entire  contents  of  one  device  (such  as  disk)  onto 
another  (such  as  tape) . 

CONVERT  TABS  to  multiple  spaces  (and  converse)  . 

DELETE  trailing  blanks  and  card  sequence  numbers,  and 
replace  with  carriage  return,  line  feed,  rubout  (for 
converting  card  images  to  paper  tape) . 

LIST  the  contents  of  any  file  oriented  device  (i.e.:  the 

names  of  files  contained  on  that  device) . 

Call  from  monitor  with  single  command. 
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TABLE  15 

FORTRAN  CHARACTERISTICS 

At  least  6-dimensional  array  capability. 

Negative  DO  LOOP  Indexing. 

Format-free  input  (Character  fields  separated  by  commas) . 
All  Printing  ASC  II  Characters  legal  in  Hollerith  Fields. 
Any  arithmetic  expression  as  a legal  subscript. 

Any  array  element  as  a legal  subscript. 
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execute.  The  loader  should  be  called  from  the  monitor  with  a 
single  instruction.  One  command  string  should  identify  the 
object  programs  to  load  and  any  desired  options  such  as  load 

maps . 

Many  loaders  demand  that  the  main  program  and  all  re- 
quired external  subroutines  be  identified  by  name  in  the  load 
command.  The  loader  should  require  only  the  name  of  the  main 
program,  and  should  search  the  files  and  load  subroutines  and 
external  subfunctions  which  have  been  identified  in  the  mam 
program.  Although  this  highly  desirable  feature  is  not  possible 
in  some  software  structures,  its  incorporation  will  significantly 
reduce  operator  error  and  machine  time  during  manual  monitor 
operation. 

If  the  loader  is  used  by  the  monitor  to  load  the 
aforementioned  utility  Routines,  its  operation  should  be 
completely  automatic.  Thus,  one  monitor  command  to  bnng-in 
the  text  editor  should  locate  the  text  editor  program,  load  it 
into  core,  and  initiate  its  execution. 

(6)  Debug  Supervisor.  A program  to  supervise  the 
execution  of  machine  language  or  FORTRAN  user  programs  for  de- 
buggine  purposes  is  highly  desirable.  The  debug  program  should 
be  called  directly  from  the  monitor,  and  should  automatically 
call  the  loader.  Once  the  user  program  is  loaded,  debug  command/ 
control  should  be  through  the  keyboard  in  an  abbreviated  natural 
language  format  where  possible.  Keyboard  control  options  should 
permit  the  operator  to  perform  the  functions  shown  in  Table  16. 

A minimum  of  four  breakpoints  should  be  definable 
prior  to  program  start  and  during  any  program  halt.  The  break 
points  are  assumed  to  be  the  address  of  a specific  instruction  in 
the  program.  It  would  be  desirable  if  the  addressed  could  be 
specified  relative  to  the  first  program  instruction  address. 

Capability  to  examine  the  contents  of  memory  words 
should  be  by  reference  to  the  symbol  name  or  address.  A variable 
name,  or  its  location  should  be  a sufficient  command  to  cause  the 
memory  word  to  be  found,  and  its  contents  typed  out.  Typically, 
the  binary  contents  of  memory  words  can  be  interpreted  in  several 
ways  (i . e . : as  an  octal  or  hexidecimal  number,  an  alphanumeric 

string,  a symbolic  instruction,  or  in  (raw)  binary).  It  is 
highly  desirable  that  the  memory  word  be  interpreted  on  output, 
and  that  interpretation  modes  be  under  operator  control. 

Control  functions  which  permit  changing  the  contents 
of  a memory  location  are  editing  functions  which  should  be 
considered  desirable,  but  not  mandatory.  The  size  of  the  Debug 
Supervisor  is  important  since  it  must  reside  in  core  with  the 
user  program Cs) . Editing  functions  should  be  omitted  if  they 
cause  a significant  increase  in  the  size  of  the  program. 


TABLE  16 

DEBUG  SUPERVISOR  CONTROL  FUNCTIONS 


j 


I 


Start  program. 

Halt  at  pre-defined  breakpoints  (minimum  of  four) . 

Examine  contents  of  registers  and  memory  words  by  symbolic 
reference. 

Make  additions  and  corrections  to  machine  language  in- 
structions using  symbolic  code  (edit  function) . 

Change  the  contents  of  registers  and  memory  words  (edit 
function) . 

Step  to  the  next  instruction. 

Continue  program  execution  to  the  next  breakpoint. 

Restart  program  from  a specified  address. 

Transfer  control  to  Debug  on  Keyboard  control  input. 


\ 
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Finally,  it  is  possible  that  a program  will  enter  a 
continuous  loop  prior  to  reaching  the  first  breakpoint.  A 
keyboard  control  input  should  be  provided  to  transfer  control 
to  the  Debug  Supervisor  at  any  time  during  user  program  execution. 
Such  manual  transfers  to  Debug  should  be  treated  as  a breakpoint 
halt,  if  possible.  This  would  make  all  breakpoint  halt  control 
functions  legal  from  a manual  transfer. 

(7)  Chain  and  Execute.  Chain  and  execute  is  a desired 
supervisory  program  that  permits  segmentation  and  execution  of 
user  programs  which  may  otherwise  be  too  large  to  reside  in  memory. 
Performance  measurement  processing  frequently  requires  lengthy 
editing  and  analysis  programs  whcih  operate  on  large  data  arrays. 
Data  arrays  as  large  as  8,000  words  are  often  required.  Floating- 
point word  formats  may  double  or  triple  data  array  sizes  in  some 
computers.  A modest  amount  of  memory  can  be  exceeded  quite  easily 
by  operating  system  and  user  program  requirements.  Program 
chaining  software  is  a cost-ef f ecitve  way  to  provide  the  needed 
capability  with  a modest  amount  of  memory. 

Chain  and  execute  systems  usually  consist  or  a resident 
main  program,  other  resident  programs,  a resident  COMMON  storage 
area,  and  a set  of  subroutines  which  can  overlay  each  other  as 
directed  by  the  operator.  These  subroutines  are  grouped  into 
Units  which  can  overlay  each  other.  One  or  more  subroutines  can 
ben  contained  ip  a Unit.  When  overlay  Units  are  loaded,  they  are 
written  on  top  of  the  core  image  area  allocated  to  overlays  and 
to  not  distrub  the  rest  of  core.  Of  course,  the  overlaying  Unit 
will  "wipe-out"  the  core  image  of  the  resident  Unit;  however,  any 
of  the  overlay  Units  can  be  restored  to  resident  core  in  their 
original  form.  Several  Unites  may  overlay  a larger  Unit  without 
overlaying  each  other.  Cascading  of  sub-overlays  should  not  be 
limited . 


An  overlay  Unit  is  loaded  into  core  when  a subroutine 
contained  therein  is  called.  It  remains  in  core  until  it  is 
over lay ed  by  another  Unit.  Any  routine  in  a resident  Unit  can 
call  a subroutine  in  a non-resident  Unit;  however,  the  calling 
program  will  be  overlayed  and  will  not  reside  in  core.  In  this 
case,  the  called  subroutine  can  only  terminate  by  calling  another 
subroutine  since  it  cannot  return  to  the  calling  program.  Resident 
subroutines  can  exist  outside  of  the  over  lay  structure,  and  can 
be  called  by  subroutines  within  the  overlay  structure  when  they 
are  in  core. 

If  the  performance  measurement  hardware  facility 
contains  more  than  32,000  words  of  memory,  the  incorporation  of 
program  chaining  software  is  not  critical. 

Measurement  Processing.  The  goal  in  defining  a candidate 
measurement  processing  software  system  was  to  achieve  a balance 
between  two  extreme  system  philosophies,  (a)  a highly  general- 
izable,  automated  system  and  (b)  a very  manual  system.  The 
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average  researcher  dreams  of  a generalizable  measurement  proc- 
essing system  which  is  automated  and  provides  him  with  inter- 
active command  and  control.  An  example  of  such  a system  would 
be  one  in  which  the  operator  would  (a)  sit  at  a keyboard/CRT, 

(b)  command  the  computer  through  all  stages  of  processing,  online , 

(c)  specify  analysis  options  by  keyboard  overlays  or  by  ligh  - 
gunning  computer  generated  lists  of  alternatives  and  td)  would  be 
provided  with  desired  hard  copy  at  his  request.  Building  a 
system  to  provide  such  operator-computer  conversation  would  take 
a large  programming  effort  and  a substantial  amount  of  hardware. 


On  the  other  hand,  the  usual  experimental  situation  consists 
of  measurement  processing  which  is  specifically  programme  or 
only  one  study,  or  narrow  range  of  studies.  As  each  new  Pro 
grammer,  researcher  or  study  problem  enter  the  environment,  most 
of  the  programming  has  to  be  re-done.  There  is  no  reason  ° 
expect  otherwise  because  each  member  of  the  research  team  wi 
brinq  into  the  environment  his  unique  background,  skill  an 
creativity.  Each  will  have  a preference  for  doing  a given  1° 
in  a unique  way.  This  leads  to  substantial  programming  for  each 
study , however,  and  it  can  decrease  the  timeliness  of  the  result 
and  increase  the  cost  of  the  product. 


Some  compromise  between  semi-automatic,  generalized  software 
and  unique  programming  for  each  study  is  necessary.  Extensive 
experience  with  several  hardware/software  systems  and  many 
simulator  and  inflight  research  studies  have  curded  us  in  the 
selection  of  processing  alternatives  which  we  believe  to  : be  the 
best  general  approach.  Certainly,  as  knowledge  of  a specific 
computer,  a specific  operating  system,  specific  hardware, and 
sDecific  set  of  research  test  plans  become  available,  some  of  the 
alternatives  Should  be  reconsidered.  Software,  like  Disneyland 
should  never  be  complete  as  long  as  there  is  creative  imagination, 


Historically,  there  are  five  stages  of  performance  measure- 
ment processing,  (1)  acquire  data,  (2)  input  data  to  processing 
media,  (3)  edit  data,  (4)  create  measures  from  ra,w  data— measure 
transformation,  and  (5)  analysis.  These  five  stages  f°™  14 

structure  of  measurement  processing  software  as  shown  in  Figur 


Each  stage  requires  specific  input  control  information  and 
data,  and  produces  specific  output.  ACQUIRE  data  reads  paper  tape 
into  computer-controlled  files  (such  as  a text,  au  i orY 
video/photographic  (V/P)  data)  and  allows  preliminary  editing. 

A label  file  containing  control  information  is  created  thr  9 
keyboard.  INPUT  arranges  and  labels  the  file  records  in  stan 
foLi,  scales  and  calibrates  data,  extracts  magnetic  tape  control 
information  from  all  files,  and  reads  the  externai magnetic  tape 

into  a file.  Raw  data  files  are  tested  for  e^r?r^in. 
final  corrections  are  made  and  data  are  placed  into  the  data  bank. 
Following  operator  directions,  measure  sets  for  each  event  are 
created  by  MEASURE  TRANSFORM  and  placed  into  a temporary  analys 
file.  Analysis’ routines  read  the  measure  sets  and  perform  op- 
erated directed  analyses.  Hard  copy  is  available  at  each  stage 
of  processing. 
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Figure, 14.  Measurement  Processing  Functions 
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(j \ ACQUIRE  Data.  The  purpose  of  this  first  proc- 
essing stage  is  to  create  computer  files  of  data  which  can  be 
input  from  paper  tape  or  keyed  directly  into  the  computer . As 
shown  in  Figure  15/  the  system  text  editor  is  used.  It  i^> 
assumed  that  a named  file  will  be  created  for  each  kind  of  data 
used  by  the  system.  For  example,  photographic  data  may  be  _ 
contained  in  the  PHOTO  file,  auditory  commands  may  be  container 
in  the  CMD  file,  etc.  Once  the  data  are  in  files,  they  shoul  i 
be  edited  for  error,  file  structure  and  format.  Editor  or  File 
Manipulator  commands  can  be  used  to  list  the  file  contents  as 
they  are  (one  line  at  a time) ; no  special  formatting  will  be 
possible  with  the  text  editor  or  file  manipulator  systems  pro- 
grams . 

In  addition  to  files  for  paper  tape  data,  a label 
file  should  be  created  at  this  time.  The  label  file  contains 
data  identification,  scaling  and  control  information  that  will 
be  used  to  format  all  data  files  in  the  next  program,  input. 

The  data  acquisition  structure  using  the  system  text 
editor  is  straight-forward;  however,  it  requires  the  generation 
of  input  paper  tapes  off-line,  without  the  benefit  of  software 
to  lead  the  data  reduction  operator  through  the  process,  and  to 
check  errors.  The  initial  plan  was  to  specify  interactive  (man- 
computer)  software  for  on-line,  computer  guided  collection  of 
non-magnetic  tape  data  (such  as  video,  auditory,  or  photographic). 
Such  a system  would  increase  the  amount  of  data  that  could  be 
collected.  More  importantly,  interactive  software  could  increase 
the  accuracy  of  the  manually  reduced  data  by  several  orders  o 
magnitiude  b y leading  the  operator  through  the  data  reduction 
process  and  by  performing  reasonability  checks  of  the  incoming 

data. 

■ 

’ Unfortunately,  an  examination  of  the  amount  of  data 
to  be  processed  reveals  that  such  a system  would  require  far 
more  computer  time  than  could  be  made  available  without  a back- 
ground/foreground (B/F)  or  time-shared  operating  system  with 
several  terminal  stations.  A larger  processing  facility  than 
has  been  defined  would  be  required  to  fully  tuilize  B/F  or  time 
shared  operations.  Even  with  a B/F  operating  system,  the  pro- 
gramming of  man-computer  interactive  software  would  require  a 
lot  of  memory  and  a substantical  programming  effort. 

A non-inter active , initial  data  reduction  system  was 
chosen  in  order  to  keep  the  facility  small,  and  to  retain  as 
much  procedural  flexibility  (ability  to  do  many  kinds  of  studies) 
with  a minimim  hardware  and  software  inverstment.  Therefore, 
the  data  would  be  punched  on  paper  tape,  offline,  then  brought 
to  the  computer  for  editing,  errors  and  all. 

The  arrangement  of  parameters  on  paper  tape  is  likely 
to  be  quite  varied  from  file  to  file  and  across  all  training 
phases.  The  data  must  be  punched  onto  paper  tape  in  an  order 
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that  is  convenient  for  the  operator  and  will  produce  the  fewest 
errors.  As  a consequence  of  the  large  number  of  parameters 
that  can  be  derived  from  many  different  sources,  it  will  be 
necessary  to  attach  a parameter  indentif ication  number  (tag) 
onto  each  measure.  More  is  said  about  this  data  formatting 
issue  in  the  next  section. 

(2)  INPUT  Data.  INPUT  arranges  and  labels  the  file 
records  in  standard  form,  scales  and  calibrates  data,  extract 
magnetic  tape  control  information  from  other  files  and  reads  the 
external  magnetic  into  a file.  The  processing  functions  are 
indicated  in  Figure  16. 


Starting  with  the  temporary  files  from  ACQUIRE,  INPUT 
performs  any  necessary  data  computation,  and  scaling.  For  data 
reduction  ease,  and  to  reduce  errors,  it  is  possible  that  data 
which  have  been  manually  reduced  may  not  be  scaled  in  units  that 
will  be  consistent  with  the  rest  of  the  data  bank.  Altitude, 
for  example,  need  be  input  only  to  the  number  of  digits  that  are 
significant  for  the  particular  maneuver  being  flown.  Units . of 
time  may  be  relative  to  a previous  time  hack;  units  may  require 
adjustment  to  mission  time.  Mils  might  be  converted  to  degrees. 
Additionally,  calibration  frames  might  provide  data  which  are 
used  to  adjust  all  data  of  a particular  parameter  to  account  for 
to  day,  or  aircraft  to  aircraft  differences.  Once  the  data  are 
properly  adjusted,  they  are  written  into  the  temporary  files  in  a 
standard  format. 


Some  of  the  data  contained  in  the  temporary  files  and 
the  label  control  file  will  be  used  to  control  initial  scaling 
and  sampling  of  digital  magnetic  tape  data.  Such  data  will  have 
to  be  identifies,  extracted,  and  placed  in  a file  of  magnetic 
tape  control  information  prior  to  reading  the  referenced  magnetic 
tape. 

The  external  magnetic  tape,  whether  it  was  generated 
in  an  aircraft,  in  a simulator,  from  some  other  source,  or  comes 
from  special  tape  conversion  equipment  will  be  dumped  entirely 
onto  disk  before  processing.  A machine  language  program  for 
this  purpose  may  be  required  unless  the  file  manipulator  monitor 
program  provides  such  dumping  capability.  It  is  likely  that 
the  data  contained  therein  will  not  be  in  standard  engineering 
units,  and  will  require  unpacking,  scaling  and  calibration. 

Next,  the  data  will  be  sampled,  and  broken  into  logical  events 
as  dictated  by  the  information  in  the  magnetic  tape  control  (MT 
Control)  and  label  files.  Records  will  be  formatted  and  written 
in  the  MT  data  file. 

All  data  files  which  will  be  subject  to  numeric  analysis 
should  be  formatted  in  a standard  way  to  minimize  programming  of 
EDIT,  MEASURE,  TRANSFORM,  and  ANALYSIS  routines.  Files  will 
contain  much  more  data  than  can  be  fit  in  memory;  generally,  pro- 
grams will  read  down  a file  and  select  only  the  data  desired.  The 
largest  record,  of  course,  has  to  fit  in  memory. 
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Figure  16.  Data  Input  Processing  Functions 


Each  data  record  should  be  identified  by  experiment, 
subject,  day,  trial,  event,  group  and  any  other  experimental 
condition  (s)  that  will  serve  as  a unique  identification  of  that 
record.  A formatting  system  that  has  been  most  successful  in 
measurement  processing  is  suggested.  It  serves  to  minimize 
search  programming;  it  permits  records  to  be  placed  in  any  order 
in  the  files,  and  it  provides  automatic  placement  (indexing)  of 
data  into  multidimensional  arrays  for  factorial  analyses.  The 
format  is  shown  in  Figure  17. 

The  data  format  requires  a fixed  size  label  record 
that  uniquely  identifies  the  following  data.  The  label  record 
usually  contains  only  numbers,  but  there  is  no  reason  why  it 
could  not  contain  both  alphabetic  and  numeric  characters.  The 
record  is  relatively  small;  typically,  20  data  words  will  be 
more  than  sufficient  for  training  experiments.  The  last  word  in 
the  data  array  contains  the  number  of  words  in  the  following  data 
record. 


The  size  of  the  data  record  is  variable  because  the  last 
element  of  the  label  array  indexes  the  read  command  for  the  data 
array.  Thus,  in  FORTRAN,  the  following  sequence, 

DIMENSION  LABEL  (20) , IDATA  (8000) 

EQUIVALENCE  (LABEL  (20),  NW) 


READ  CIUNIT)  LABEL 

READ  (IUNIT)  (IDATA (I) , 1=1,  NW) 


would  cause  a binary  (unformatted)  read  of  the  label  record  with 
its  unique  identifiers  and  the  number  of  words  in  the  data  array, 
followed  by  a read  of  that  data  array.  The  data  array  must  be 
dimensioned,  however,  to  the  largest  anticipated  size.  Typical  y, 
the  data  array  has  been  treated  as  a one-dimensional  array  tor  . 
input/output  (I/O)  purposes.  It  can  be  equivalenced  to  any 
combination  of  smaller  single  or  multi-dimensional  arrays,  and/or 
scalar  variables. 

The  problem  of  identifying  the  parameters  within  the  data 
array  requires  special  attention.  Any  one  data  source  for  a 
given  phase  of  training  will  produce  a relatively  small  number 
(10-20)  of  parameters.  Across  all  training  phases  and  all 
sources  of  data,  however,  the  parameter . list  might  be  quite  large 
(100-200).  Not  only  is  the  parameter  list  large,  but  the  order 
in  which  parameters  will  be  contained  in  any  file  is  likely  to 
be  varied.  The  method  of  identifying  parameters  must  be  carefully 
selected  to  minimize  the  programming  and  processing  of  house- 
keeping functions  throughout  the  measurement  processing  system. 

There  are  at  least  two  good  methods  to  uniquely  identify 
parameters  in  the  data  array.  First,  all  parameter  organization 
subsets  can  be  specified  in  a look-up  table  that  is  contained  in 
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all  subsequent  programs.  One  element  of  the  record  label  can 
contain  a pointer  to  the  portion  of  the  table  that  defines  the 
parameter  ordering  in  the  following  data  array.  A second  method 
would  be  to  pack  the  data  value  with  the  parameter  number.  Each 
reference  to  data  in  subsequent  programs  would  require  unpacking 
to  determine  the  parameter  number  and  its  value. 

The  choice  of  method  cannot  be  made  until  the  hardware  is 
selected  because  it  will  be  dependent  on  (1)  the  number  of  binary 
digits  in  a computer  word,  (2)  subroutine  access  time,  (3)  ^-lXed 
and  floating  point  arithmetic  processing  time,  and  (4)  whether 
the  data  are  stored  in  fixed  or  floating  point. 

The  number  of  binary  digits  dictate  how  large  a.  number  can 
be  prepresented  in  fixed  point  format?  if  it  is  sufficiently 
large  C24  bitsl,  then  a parameter  identification  tag  can  be  packed 
with  the  data  in  the  same  word  and  still  retain  four  to  five  digit 
accuracy.  Unpacking  of  data  will  be  a highly  repetative  operation 
which  can  be  performed  with  a machine  language  subroutine  or 
integer  mode  arithmetic.  The  amount  of  time  consumed  by  each 
operation  will  be  an  important  factor.  If  the  data  must  be  stored 
in  floating  point  to  achieve  accuracy  (because  of  an  insufficient 
word  size) , parameter  tag  packing  with  the  data  word  will  probably 
not  be  desirable. 

Processing  would  be  most  straight- forward  if  the  data  were 
stored  in  a binary,  24-bit  word.  This  would  permit  integer  mode 
manipulations  (often  faster  than  floating  point)  and  would 
probably  eliminate  the  necessity  of  a table  look-up. 

(3)  EDIT.  Raw  data  files  are  tested  for  error  in  EDIT, 
where  final  corrections  are  made  and  data  are  placed  into  named 
files  in  the  data  bank.  It  is  assumed  that  all  textual  data  will 
be  edited  using  the  monitor  text  editor  and  will  not  be  edited  in 
this  program. 

Editing  capability  should  include  ability  to  correct 
(a)  numeric  data  within  a record,  (b)  labeling  data  within  the 
label  record,  and  (c)  record  manipulations  such  as  combining , 
splitting  and  deleting  records  from  the  file. _ It  is  possible, 
also,  for  the  wrong  labels  to  be  associated  with  the  wrong  data 
records.  Typically,  when  this  happens  all  labels  are  misplaced 
by  one  record  or  several  records  and  need  to  be  "rippled  up  or 
back."  Such  editing  will  take  place  during  record  editing  (as 
opposed  to  data  editing) . 

EDIT  is  a man-computer  interactive  program  which  will 
require  the  operator  or  experimenter  to  make  data  corrections 
while  on-line.  The  operator  will  lead  the  computer  through  all 
corrections  and  computations,  although  the  actual  manipulation . 
and  housekeeping  will  be  performed  by  the  computer  (a  substantial 
chore).  All  operator  input  has  to  be  error  checked  and  the  soft- 
ware at  every  operator  input  point  must  provide  instructions  on 
entry  options,  formats,  and  diagnostic  error  messages.  The 
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processing  functions  are  shown  in  Figure  18.  The  operator  control 
functions  associated  with  each  processing  function  are  shown  in 
Table  17. 


The  first  processing  function  in  EDIT  is  to  open  the 
file  to  be  edited  (by  name)  and  to  find  the  first  record  to  be 
edited.  (The  first  record  of  the  file  will  be  the  first  record 
to  be  edited  only  at  the  beginning  of  data  editing).  Operator 
specification  of  the  desired  record  should  be  by  subject,  date 
trial  and  event  rather  than  record  number.  The  record  control 
module  will  perform  the  necessary  housekeeping  and  manipulation 
functions  to  search  for  the  new  data  records  and  set-up  the 
correct  data  bank  file.  In  pre-positioning  the  data  bank  file 
it  would  be  desirable  to  read  up  to  the  requested  record  and  be 
able  to  write  corrected  records  from  that  point,  on. 

In  the  file-oriented  mode  of  the  handlers  it  may  or 
may  not  be  possible  to  open  a file  for  both  reading  and  writing. 
The  data  bank  file  may  have  to  be  copied  onto  a scratch  disk, 
then  written  from  the  scratch  disk  onto  the  data  bank  disk  up 
to  the  indicated  record  in  order  to  have  the  data  bank  file 
opened  for  writing  and  properly  positioned.  Since  the  files 
are  likely  to  be  quite  large,  inability  to  open  a file  for  both 
reading  and  writing  tends  to  dictate  a strong  desirability  for 
at  least  three  logical  disk  units  (possibly  four  logical  units) . 


« 


Next,  the  data  are  checked  for  errors  against  criteria 
that  are  contained  in  the  error  criteria  file  and  preliminary 
statistics  (mean,  standard  deviation,  min  and  max)  are  computed 
for  repetitive  parameters.  If  error  criteria  are  to  be  changed 
(as  they  will  during  initial  data  collection  for  each  study) , 
a switch  option  to  key  in  such  criteria  should  be  included. 


Historically,  two  kinds  of  error  criteria  have  been 
quite  useful.  First,  data  should  be  checked  against  the  maximum 
and  minimum  values  expected  for  any  parameter  in  the  event  being 
measured.  It  can  be  seen  that  error  criteria  tables  will  have 
to  be  constructed  for  each  reasonable  class  of  events,  and  that 
the  event  number  in  the  label  record  can  indicate  which  set  of 
criteria  to  use.  Secondly,  where  successive  samples  of  the  same 
parameter  are  contained  in  the  data  record,  each  parameter  should 
be  checked  against  criteria  for  the  maximum  absolute  magnitude 
change  that  is  anticipated  from  sample  to  sample.  These  two 
checks  have  caught  most  data  errors  in  past  experience  when  the 
criteria  were  empirically  set  to  false  alarm  (error  flag  a good 
number)  about  1%  of  the  time.  Preliminary  statistics  or  re- 
petitively sampled  data  within  one  record  will  provide  a good 
indicator  of  where  the  error  criterion  chould  be  set. 

The  highest  source  of  error  is  expected  to  be  the 
manually  reduced  data.  Transposition  errors  are  expected  to 
account  for  about  40??  of  these  errors.  Transposition  errors  in 
the  most  significant  digit  stand  a good  chance  of  being  caught 
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I/O  CONTROL 

Page  control. 

Record  control. 

Hard  copy. 

Enter  edit  mode  to: 
edit  data 
edit  records 
delete  scans 
Refresh  page. 

Retest  data. 

Exit  and  copy-out  files. 

EDIT  MODE  CONTROL 

* 

Mean. 

Linear  fit  from  previous  or  following  data. 
Type  in  correction. 

Delete  data. 

Flagged  datum  is  good. 

Label  value  change. 

Retest  data. 


RECORD  EDIT  CONTROL 

Combine  records. 

Split  records. 

Delete  records. 

Ripple  up/down  tables . 
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by  the  absolute  magnitude  of  change  criterion.  However, 
transposition  errors  in  the  least  significant  digits  will  be 
very  difficult  to  detect  without  substantial  knowledge  of  the 
statistical  properties  of  the  particular  parameter  and  its  past 
history.  The  number  of  data  digits  in  any  one  number  that  is 
keyed-in  by  an  operator  during  ACQUIRE  should  be  kept  to  a 
minimum  to  decrease  the  probability  of  an  initial  error  and  to 
increase  the  probability  that  the  error  checking  routine  will 
detect  one. 

/ 

/ 

Next,  the  data  are  displayed  to  the  operator  oh  a CRT. 
Each  record  is  likely  to  produce  many  pages  of  data.  /Each  page 
should  contain  the  data  labels  in  a header  that  is  adequately  de- 
coded. The  header  for  subject  1,  day  2,  trial  5,  event  22  should 
at  least  show  "S  1 DY  2 TR  5 EV  22"  (instead  of  just  "12522")  in 
addition  to  the  page  number  and  the  total  number  af  pages.  All 
data  parameters  shown  within  the  page  should  be /labeled  with 
parameter  name  abbreviations,  where  possible.  /The  last  page 
should  include  summary  statistics  of  each  repetative  sample. 

Figure  19  shows  an  example  of  such  a page  fonrtat  for  a data  editor 
which  is  under  development.  / 

. . / 

The  I/O  control  brancing  module  permits  the  operator 
to  control  the  functions  shown  in  Table  17  through  abbreviated 
natural  language  commands  to  the  keyboard.  The  operator  should 
be  able  to  page  foreward  or  backward,  one  page  or  n-number  of 
pages  within  a record.  He  should  be  able  to  advance  or  back-up 
n-number  of  records  with  the  file.  He  shouljd  be  able  to  request 
hard  copy  of  any  page.  Ability  to  refresh  the  page  on  command, 
and  retest  the  data  for  errors  should  also  be  provided.  The 
software,  however,  should  automatically  recompute  the  statistics 
prior  to  displaying  them  on  the  last  page  if  a change  has  been 
made  in  a preceeding  page.  The  operator  should  also  be  able  to 
exit  the  EDIT  program  at  this  point,  causing  the  data  files  to 
be  properly  altered  by  any  changes  and  closed.  The  status  of  the 
file  editing  should  also  be  printed  by  such  a command,  indicating 
the  first  and  last  record  edited.  In  addition  to  I/O  control, 
this  module  should  provide  access  to  the  data  edit  module;  the 
command  to  enter  edit  should  imply  the  purpose  for  entering  the 
edit  module  (such  as  edit  data,  edit  records  or  delete  scans)  to 
conserve  input  requests. 

Entry  into  the  edit  mode  should  cause  the  software  to 
automatically  search  for  flagged  errors  and  request  a keyboard 
command  to  perform  one  of  the  following  options;  (a)  replace 
the  number  with  the  average  of  the  preceeding  and  following 
numbers,  (b)  replace  the  number  with  a linear  fit  of  the  preceeding 
two  numbers  or  following  two  numbers,  (c)  type  in  a number,  (d) 
delete  the  datum  or  (e)  delete  the  error  flag,  indicating  that  the 
datum  is  good.  In  addition  to  changing  flagged  errors,  the  op- 
erator should  be  able  to  exercize  the  edit  options  on  any  manually 
designated  datum.  Editing  of  label  values  should  also  be  possible. 
It  should  also  be  possible  to  delete  whole  lines,  or  scans  of 
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Figure  19.  Sample  Display  Page  Format. 

(Previous  pages  contained  data  lists  up  to  Scan  97. 
The  software  from  which  the  table  was  derived 
permitted  variable' sample  rates  for  each  parameter.) 
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data  (.a  scan  is  one  sample  of  all  parameters  taken  at  the  same 
time).  Exit  from  the  edit  mode  to  the  I/O  branching  module 
should  be  a manually  commanded  function  except  where  scans  are 
deleted.  Deleting  scans  should  automatically  branch  to  re- 
computation of  record  statistics  and  re— display  of  the  page. 

A specially  formatted  edit  mode  display  page  should 
be  constructed  to  show  only  a sample  of  the  parameter  being 
edited  and  an  indication  of  the  datum  being  operatied  on.  The 
edit  mode  page  should  provide  feedback  of  each  correction. 

The  remaining  editing  operation  is  record  edit.  Here 
the  operator  should  have  the  capability  of  (a)  combining  records, 
(b)  splitting  records,  (c)  deleting  records,  and  (d)  rippling 
labels  up  or  down  the  records.  Record  combining  should  cause 
two  adjacent  records  to  be  brought  together  into  the  first  recor  , 
and  the  label  for  the  second  record  should  be  either  deleted,  or 
all  labels  should  be  rippled-up.  If  the  INPUT  program  has 
mistakenly  placed  data  into  two  records  instead  of  one,  it 
probably  also  has  mis-registered  the  labels.  The  command  string 
de-fault  assumption  should  be  that  the  labels  are  out  of  order 
and  should  be  rippled-up  (record  #2  label  goes  to  record  #3, 
etc.).  The  converse  error  can  also  occur:  INPUT  could  have 

placed  data  into  two  records  instead  of  one.  In  this  case, 
records  should  be  split,  and  the  command  string  de-fault  assump- 
tion should  be  that  the  labels  should  be  rippled-back  unless 
otherwise  directed.  The  same  applies  to  deleting  records.  How- 
ever, when  labels  are  rippled  up  or  back,  there  will  either  be 
one  more  or  one  ldss  label  set  than  data  records.  The  next 
action  for  the  operator  will  be  to  add  the  missing  label,  or 
delete  the  extra  label  from  the  file. 

Although  it  is  not  shown  on  the  flowchart,  there  should 
be  a switch  capability  to  provide  automatic  error  checking  and 
listing  of  data.  Automatic  sequencing  through  record  read,  test 
data,  and  list  data  records  (with  error  flags)  should  continue 
until  the  record  file  is  exhausted  or  the  switch  is  turned  off. 


The  design  and  programming  of  the  edit  routine  will 
require  careful  planning  and  thought.  However,  the  utility  of 
such  a program  cannot  be  overstated  since  the  formatting  and 
correcting  of  data  files  historically  consumes  the  most  time  in 
measurement  processing. 


(4)  MEASURE  TRANSFORM.  Having  formatted  and  corrected 
data  on  file,  the  next  stage  is- to  create  performance  measures 
from  the  raw  data,  and  to  output  these  measures  in  a temporary  _ 
file  that  will  be  used  for  analysis.  MEASURE  TRANSFORM  processing 
functions  are  shown  in  Figure  20.  Fundamentally,  the  processing 
involves  reading  a raw  data  record  from  the  data  bank,  and 
converting  the  raw  data  into  specifically  designated  measures. 
Instructions  for  creating  each  measure  are  contained  in  a measure- 
ment request  file. 
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Figure  20.  Measure  Transform  Processing  Functions 
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It  is  assumed  that  the  measurement  request  file  will  be 
created  and  changed  with  the  monitor  text  editor.  Although  the 
specific  construction  of  the  file  will  depend  on  the  design  of 
the  program,  one  way  of  computing  the  required  functions  is 
implied  by  the  file  contents  shown  in  Table  18.  The  first  file 
entry  should  indicate  to  the  program  the  number  of  measure  sets 
that  are  to  be  processed.  For  each  measure  set  the  first  set  of 
entries  should  include  the  event  number  that  is  to  be  processed, 
the  name  of  the  data  file,  and  the  number  of  measures  requested. 

Each  measure  request  should  define  the  parameter  or 
parameters,  the  measurement  interval,  the  type  of  transformation, 
and  the  location  of  control  variables  or  their  values . Although 
most  processing  will  involve  only  one  parameter  at  a time,  multi- 
dimensional algorithms  such  as  correclations  or  bode  plots  will 
require  simultaneous  processing  of  more  than  one  parameter.  Note 
that  the  measurement  request  file  will  be  a direct  translation  of 
the  measurement  specifications  for  each  training  phase  contained 
in  the  Phase  III-C  report. 

The  measurement  interval  is  defined  by  indicating  the 
conditions  under  which  measurement  is  to  start  and  stop.  Measure- 
ment can  simultaneously  start  and  stop  if  only  one  value  is  de- 
sired. For  example,  pitch  attitude  at  liftoff  can  be  defined  by 
starting  and  stopping  measurement  when  the  weight-off-wheels 
discrete  changes  from  zero  to  one,  or  when  the  rate  of  climb  is 
greater  than  100  f eet-per-minute . 

The  type  of  transformation  to  be  computed  within  the 
measurement  interval  (between  start  and  stop)  can  be  value  (such 
as  pitch  attitude  at  takeoff) , an  error  score  (such  as  the  differ- 
ence between  the  assigned  altitude  and  the  actual  altitude) , or 
any  one  of  several  time,  amplitude,  or  frequency  domain  trans- 
formations shown  in  Table  19. 

Algorithm  control  variables,  such  as  start  and  stop 
conditions,  tolerance  band  values,  and  error  history  overlays 
can  reside  in  the  record  bei'ng  processed  (such  as  weight-off- 
wheels)  , other  data  files  (such  as  auditory  command  values),  or 
they  may  not  be  contained  in  the  data  base  at  all.  Each  measure 
request  must  indicate  where  to  find  each  algorithm  controlvariable 
This  is  done  by  indicating  the  name  of  the  file  and  the  name  or 
number  of  the  parameter  to  look  for.  Since  all  data  are  formatted 
by  subject,  day,  event,  etc.,  the  software  knows  the  record  being 
processed  and  can  find  the  corresponding  record  in  another  file. 
Special  cases  may  arise,  however,  where  control  data  are  not 
contained  in  the  files,  and  the  measure  request  set  will  have  to 
provide  the  control  data. 


I 


' 


B i 


TABLE  J8 

MEASURE  REQUEST  FILE  CONTENTS 

3.  Number  of  requested  measure  sets. 

2.  For  each  request  set:* 

a.  Legal  event  (event  which  is  to  be  processed) . 

b.  Name  of  data  file. 

c.  Number  of  measures  requested: 

(1)  Parameter  (or  parameter  for  multi-dimensional 
algorithms). 

(2)  Measurement  interval  (start/stop) . 

(3)  Type  of  transform. 

i.e.:  Value,  Error  score,  and  each  Time/ 

Amplitude/Frequency  domain  treatment. 


(4)  File  location  of  algorithm  control  variables. 


TABLE  19 


CANDIDATE  MEASURE  TRANSFORMS 


VALUE  (At  specific  times,  points,  -functionals). 

-Counts- 

TIME  HISTORY  (From  specific  times,  points,  [functions]  to 
specific  times,  etc.). 

[Three  ways  of  describing  time  history:] 

1 . Time  Domain 

TOT,  I,  in/out  TOL 
T to  DO 

2.  Amplitude  Domain 

Central  tendency 
Avg,  AA,  Med,  Mode 
Variability 

Rng,  Min,  Max,  RMS,  SDEV 

3 . Frequency  Domain 
Reversals 

Auto  Correlation  (periodocity) 

Harmonic  analysis  (power  spectra) 

I/O  analysis  and  models  (multi-dimensional  algorithm) 
Bode  plot  (amplitude  £ phase  plots) 

Root  locus 
Transient  response 

ERROR  HISTORY  OVERLAYS 

ATime  history  (and  various  treatment  and  desired  values) . 
MULTI- DIMENSIONAL  ALGORITHMS 

Correlations  between  measures  at  a fixed  points,  (x  vs  y) 
Lagged  cross-correlations  (crude  phase  meas.).  (x  vs  y 
at  later  time) . 

Multiple  regressions  (aX  + bY  + cZ  vs  W) . 

Canonical  correlation  (aX  + bY  + cZ  vs  dU  + eV  + fW) . 
Multi-dimensional  system  models  (multi-parameter, 
dynamic  rep.). 


Measure  transformation  program  functions  start  with  a 
reading  of  the  measure  request  file  (Figure  20)..  The  first  in- 
struction read  from  the  file  is  the  number  of  measure  sets 
(events)1  to  be  processed.  Next,  the  legal  event  number,  data 
file  name  and  the  number  of  measures  in  the  set  are  read.  The 
data  file  can  then  be  opened  and  searched  for  the  first  record 
of  the  requested  event.  When  it  is  found,  all  measures  pertaining 
to  that  data  record  are  generated  by  processing  each  measure 
request  sequentially  until  all  measures  in  the  set  have  been 
computed  and  placed  in  the  measures  data  array. 

The  measure  processing  loop  starts  with  a definition 
of  control  variables  from  the  measure  request  file.  Named  files 
are  searched  for  unsatifsied  (undefined)  algorithm  control 
variables.  Next,  the  specified  parameter  is  sampled  until  the 
start  conditions  are  met.  Sampling  and  computation  continues 
until  stop  conditions  are  met.  At  this  time  any  residual 
computations  are  made  and  the  computed  measure  is  placed  in 
a measures  data  array.  When  all  measures  are  computed,  the  label 
of  the  data  bank  record  is  used  to  uniquely  identify  the  measure 
data  (except  for  the  number  of  words  counter),  which  are  written 
into  a temporary  measurement  file. 

The  above  process,  starting  with  C in  Figure  20,  is 
repeated  until  all  data  records  have  been  read. 

If  there  is  more  than  one  measurement  set  requested 
(more  than  one  event) , the  entire  process,  starting  with  B in 
Figure  20,  is  repeated  until  all  measurement  sets  have  been  ‘ 
processed  and  listed  in  the  line  printer. 

In  operation,  it  is  envisioned  that  the  experimenter 
will  formulate  the  measurement  request  file  for  trial  data, 
receive  preliminary  output,  and  iteratively  change  the  file  until 
the  measurement  output  appears  reasonable.  Then  he  will  call  in 
the  analysis  programs . 

(5)  ANALYSIS . The  last  stage  in  measurement  proc- 
essing is  to  perform  statistical  analyses  on  the  measures  which 
have  been  placed  in  temporary  files  by  the  measure  transform 
program.  The  analysis  program  (Figure  21)  will  receive  its  in- 
structions from  an  analysis  control  file,  and  will  read  all 
records  of  the  temporary  measures  file,  one  measure  at  a time, 
and  perform  the  desired  analyses  by  calling  them  up  as  subroutines. 
The  analysis  control  file  should  contain  the  information  shown  in 
Table  20. 


10ne  measure  set  is  assumed  per  event.  An  event  is  assumed  to  be 
the  basic  unit  of  analysis  which  may  or  may  not  correspone  to  a 
training  phase.  It  is  desirable  to  retain  flexibility  for  each 
experimenter  to  define  an  event. 

. ! 
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TABLE  20 

ANALYSIS  CONTROL  FILE  CONTENTS 


CONTROL  VARIABLES  FOR  EACH  ANALYSIS: 

CD  Event  number. 

C21  Conditions  to  be  analyzed  (S,  D,  Trial,  etc.). 

(3)  Measure  Definition  Cwhere  to  find  it  in  MREQ) . 

(a)  Single  measured. 

Cbl  Multiple  measures  for  multi-dimensional 
analysis . 

0D  Analysis  subroutine  to  call. 

C5 1 Analysis  criteria  Ccritial  F- ratios,  correlations 
etc.}.  > 

1 

C6)  Output  labeling. 


The  first  control  variable  is  the  event  number  which 
is  needed  because  more  than  one  event  may  be  contained  in  the 
MREQ  and  TEMP  files.  Event  number  will  be  used  also  to  control 

output  labeling. 


The  conditions  to  be  analyzed  should  be  specified  next 
in  a way  that  permits  the  analysis  of  all  factorial  conditions 
or  separate  analyses  of  special  conditions.  Often,  ± ch 

problems  demand  capability  to  perform  sub-analyses  of  data  which 
have  been  collapsed  or  broken-out  in  special  ways. 


Complete  measure  definitions  are  contained  in  the  MREQ 
file-  the  analysis  control  file  need  only  reference  where  in 
MREQ  * the  desired  measure  is  defined.  Multi-dimensional  analyses, 
however,  relate  two  or  more  measures.  Measure  definition  must 
have  the  capability  to  specify  more  than  one  measure  for  one 
analysis. 


Analysis  subroutines  should  be  indicated.  Analysis 
criteria,  such  as  critical  F-ratios  and  significant  correlation 
values  should  be  entered.  This  is  especially  important  for 
efficient  processing  where  successive  sub-analyses  are  predicted 
on  the  results  of  a previous  analysis  (as  m tests  for  simple 
effects  or  in  multiple  discrimination  analysis) . 


Finally,  output  labeling  should  be  implied  by  the 
aforementioned  control  variables;  however,  if  they  are  not 
completely  defined,  there  should  be  a table  entry  for  them. 


The  last  record  of  the  analysis  control  file  should 
contain  a coded  entry  to  indicate  that  all  analysis  requests 
have  been  satisfied. 


Having  set-up  the  analysis  control  file,  one  set  of 
instructions  will  be  read  (Figure  21).  Unless  the  exit  code 
is  encountered,  the  needed  definition  of  the  measures  ^UGh 
where  the  measures  are  in  the  temporary  file)  will  be  read  from 

the  MREQ  file. 


Next,  the  temporary  file  is  processed.  Notice  th 
data  transposition  occurs?  one  measure  (datum)  wl11  ke^en 
from  each  record  to  fill  the  smallest  cell  in  an  n-dimensional 
analysis  array.  (For  analyses  which  required  two  measures,  two 
measures  will  be  taken).  The  position  of  the  datum  in  the 
analysis  array  will  be  indexed  by  the  contents  of  the  label 
arrav  for  that  record.  The  analysis  array  should  be  filled  when 
the  last  record  of  the  temporary  file  is  read.  The  file  will  be 
closed  and  read  again  to  process  the  next  measure  at  the  co 
elusion  of  the  analysis  of  the  first  measure.  Using  the  label 
array  to  index  the  analysis  array  in  this  fashion  e lemma tes  t e 
requirement  for  temporary  file  records  to  be  written  in  y 
special  order.  The  data  are  extracted  and  properly  disposed 
as  they  a^e  encountered. 
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. . t^ie  data  are  ready  for  the  candidate  analysis 

as  subroutines  21  * The  analysis  programs  are  envisioned 

as  subroutines  that  will  reside  in  a re-locatable  overlay 

hf^  (dlsc^3sed  undfr  the  topic  of  the  executive  monitor, 
chain  and  execute  supervisor).  Using  such  re-locatable  over- 

11  statements.f°r  logical  groupings  of  analysis  sub- 

hra^ihfnnC?n  Contained  in  the  main  program,  with  control 
is  can  2d  t°  th®  fegueated  analysis.  As  the  analysis  routine 
' . **  y1!!  bo  brought  into  memory  for  processing.  It 

So2°2nreall?t; LC  t0  eXpeCt  aU  anal^sis  routines  to  be  llied 
aroun?™  pr°?ram:  However,  the  use  of  overlays  permits 

g ouping  Of  the  subroutines  that  require  similar  data 

III™' ZL11  reduce  number  of  special  purpose  pro- 
grams that  have  been  written . 


_ Analysis  subroutines  should  perform  as  much  book- 

keeping as  possible.  For  example,  many  canned  analysis  of 
variance  subroutines  output  f-ratios,  but  do  not  test  the  f- 
ra  l os  against  the  critical  ratio  because  they  are  built  as 
general  purpose  subroutines . This  requires  the  experimenter 

analv?^h  ^ !' lgnif iCant  results'  and  requires  an  additional 
analysis  pass  to  perform  multiple  range  tests,  etc.  Analysis 

variance  subroutines  should  contain  an  option  to  test  the 
ratios  against  an  experimenter  supplied  (in  ANCT)  critical 

i2te?ar??2nOUtSUt-theaSignifiCant  leVel  °f  each  main  effect  and 
^ V Having  done  so,  the  multiple  range  tests  (tests  of 

mpie  effects)  can  be  processed  automatically  on  the  basis  of 

result,  and  tables  of  means  and  standard  deviations 

Su2h  33  3 fUnCtl°n  of  a11  significant  effects. 

automatic  processing  should  serve  to  provide  for  the  ex- 

perimenter  the  output  he  needs  with  the  fewest  possible  passes 
on  the  computer.  However,  automatic  processing  should  not  prevent 
the  experimenter  from  specifying  the  output  he  w?v'ts. 


•e  General  purpose  analysis  routines  fre  j ently  do  not 

per  orm  the  exact  computation  that  the  experimenter  wants.  Since 
experimenter  is  the  one  responsible  for  the  results  (not  the 
computer  program)  there  is  no  alternative:  either  the  experi- 

menter or  a programmer  available  to  him  must  re-program  or  write 
a new  routine  that  suits  the  study  requirements.  Often,  a few 
nours  of  programming  can  provide  the  necessary  capability.  It 

exPeJted  that  variations  in  the  analysis  routines  will  be 
progremmed  from  study  to  study.  As  studies  are  conducted,  a 
substantial  library  will  be  built.  As  the  library  is  built, 

willP^°faablllty  that  an  existin9  routine  will  fit  the  experiment 


. A^lysis  routines  should  label  their  output  with 

c£aractars  that  ara  related  to  the  experiment  being 
conducted.  General  purpose  routines  often  label  data  with 
^^rary  notation,  ^such  as  r,  J,  K,  L,  M,  etc.  The  experimenter 
others  who  are  interested  in  the  results  of  the  experiment 
must  go  to  another  source  to  find  out  what  each  of  these  labels 
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TABLE  21 


CANDIDATE  ANALYSIS  SUBROUTINES 
FACTORIAL  BREAKOUTS 

Average,  Absolute  average.  Median,  Mode,  Standard 
deviation. 

CORRELATION 

MULTIPLE  REGRESSION 

Linear 

Non-linear 

CANONICAL  CORRELATION 
FACTOR  ANALYSIS 

ANALYSIS  OF  VARIANCE  (At  least  4-factor) . 

Randomized  blocks. 

Treatment  X subjects. 

Within  and  between. 

Partially  heirarchal. 

TESTS  OF  SIMPLE  EFFECTS 

Duncans  test 
Newman-Keuls 

ANALYSIS  OF  CO- VARIANCE 

T-TEST 

F-TEST 

NON- PARAMETRIC  TESTS 
CURVE-FITTING  (Least  squares  fit)  . 
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means  for  that  experiment.  While  this  is  not  a big  issue,  it  is 
simple  enough  to  set-up  a file  of  experimental  condition  labels 
that  imply  the  conditions  (such  as  S for  Subject,  D for  Day,  T 
for  Trial,  R for  Replication,  etc.}  and  to  use  these  labels 
hroughout  the  processing  and  analysis.  It  is  a simple  matter, 
for  example,  to  program  an  analysis  of  variance  subroutine  to 
expect  such  labels  to  be  in  common  storage,  or  in  the  call  state- 
ment. A default  label  assumption  could  be  made  within  the  program 
if  the  labels  were  not  set-up. 


Alpha-labeling  applies  to  other  output  as  well.  For 
example,  measure  number  22  would  be  more  meaningful  if  it  were 
output  as  "ALT  SDEV,"  (for  altitude  standard  deviation)  instead 
of  measure  22.  Similarity,  event  14  might  be  better  labeled, 
"TAKE-OFF  ROLL." 

Data  output  should  be  formatted  so  that  it  can  be 
transfered  onto  a photo-ready  page  for  a report.  Much  report 
preparation  time  is  spent  re-formatting  and/or  retyping  data. 

Proper  attention  to  the  output  format  can  sace  considerable  work 
later  on. 

As  a concluding  comment,  all  programs  except  EDIT  provide 
operator  process  control  through  control  files.  All  control 
filos  contents  could  be  set-up  offline  with  card  decks  rather 
than  online  with  the  text  editor.  The  first  instruction  in  all 
processing  stages  would  be  to  read  the  control  cards  into  the 
file.  Although  quite  acceptable  (even  preferred)  in  a large, 
batch  processing  computer  facility,  the  use  of  cards  for  control 
f files  would  extend  the  overall  processing  time  because  of  the  ? 
time  required  to  get  on  a keypunch  machine,  re-order  the  deck 
and  re-submit  the  program  between  each  run.  Although  there  are 
times  when  the  experimenter  will  need  time  to  study  results, 
during  initial  measurement  development,  days  v;ill  be  saved  if 
the  experimenter  can  quickly  move  from  trial  output  to  changes 
in  the  request  files,  and  back  into  the  programs.  The  turn- 
around time  is  a critical  factor  in  the  time  required  to  develop 
the  measure  set,  and  analysis  routines. 
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IV.  THE  PERSONNEL  SUBSYSTEM 


Phase  IIIC  Design  Studies 

In  the  previous  phase  report  (Phase  IIIC:  Design  Studies), 
an  analysis  was  made  of  personnel  requirements  for  the  perform- 
ance measurement  system.  In  that  analysis,  estimates  were  given 
of  (1)  the  types  of  skills  needed  in  the  system  and  (2)  the 
number  of  individuals  required  for  system  staffing. 

As  was  pointed  out  in  the  previous  report,  system  personnel 
requirements  stem  from  required  manual  tasks  and  the  quantity  of 
required  manual  data  processing.  In  the  two  preceding  chapters 
of  the  present  report,  additional  clarification  has  been  given  of 
task  requirements  within  the  data  acquisition  and  data  processing 
subsystems.  It  is  reasonable,  therefore,  to  re-assess  the  total 
personnel  requirements  of  the  performance  measurement  system. 

Qualitative  Personnel  Requirements 

Consistent  with  the  previous  personnel  requirements  analysis, 
it  is  possible  to  identify  six  skill  types  needed  for  effective 
operation  of  the  performance  measurement  system:  (. ) a system 

director,  (2)  research  personnel,  (3)  a computer  programmer, 

(4)  data  clerks,  (5)  engineers  and  technicians,  and  (6)  a 
secretary.  A general  job  description  can  be  given  for  each  of 
these  skill  classes. 

1.  System  director.  One  individual  must  be  responsible  for 
management  and  supervision  of  the  operating  system.  The  success 
of  a system  of  this  type  depends  to  a very  large  extent  upon 
careful  planning  and  continuous  control.  Maximum  system 
effectiveness  can  only  be  achieved  by  a detailed  program 
technical  plan  and  considerable  adaptability  in  program  execution 
when  the  unexpected  occurs  (as  it  frequently  does  in  complex 
applied  experimentation) . Among  other  skills,  this  individual 

is  expected  to  have  knowledge  and  experience  concerning  (1)  combat 
mission  flying  for  all  appropriate  aircraft  types,  (2)  military 
flight  training  procedures,  (3)  applied  experimental  design  , 

(4)  computer  data  processing,  and  (5)  data  analysis,  interpreta- 
tion, and  report  preparation.  "*■ 

2.  Research  personnel.  Trained  scientific  personnel  are 
necessary  to  support  all  phases  of  the  performance  measurement 
process  ranging  from  experimental  planning,  conduct  of  experi- 
mental studies,  to  final  report  preparation.  During  the  actual 
conduct  of  experimental  studies,  the  research  specialist  will  be 
present  at  mission  briefings  to  extract  necessary  information 
relevant  to  performance  measurement  (see  page  25) . In  some  cases, 
he  will  be  expected  to  serve  an  an  inflight  experimenter  (see 
page  20) , and  will  be  responsible  for  the  operation  of  the  RCD 
units.  The  research  specialist  will  be  responsible  for  super- 
vision of  data  processing  for  his  assigned  flights,  and  may 
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perform  measurement  debriefings  for  instructor  anc}  student 
pilots.  The  research  specialist  is  expected  to  be  a graduate 
behavioral  scientist  with  knowledge  in  depth  of  military  missions 
and  flight  training  as  well  as  applied  behavioral  experimentation. 

3.  Computer  programmer.  As  has  been  pointed  out  in  Chapter 
III  (pages  46-47)  , a compromise  approach  has  been  taken  in  the 
design  of  the  data  processing  subsystem  to  provide  generalized 
software  with  the  assumption  that  some  unique  programming  for 
each  individual  study  will  be  required.  It  has  been  our  con- 
sistent experience  that  this  approach  offers  the  greatest 
flexibility  and  overall  effectiveness  for  performance  measurement 
systems.  It  requires,  however,  the  presence  of  a computer 
programmer  both  to  set  the  specific  study  software  and  to  insure 
that  the  data  processing  programs  are  running  properly.  For 
inflight  and  simulator  experimentation,  this  skill  type  has 
consistently  proven  to  be  an  essential  member  of  the  experimental 
team. 

4.  Data  clerks.  The  data  acquisition  subsystem  provides  a 

heterogeneous  set  of  input  data  which  must  be  processed  by  data 
clerks.  Table  10  (page  31)  has  listed  the  major  personnel  tasks 
required  for  manual  data  processing  of  video,  digital,  audio,  and 
photo  raw  data.  As  noted,  video  and  photo  data  present  very 
similar  manual  data  transcription  problems.  With  video,  photo  or 
audio  data,  the  end  product  from  the  data  clerks  is  a punched 
tape.  Very  detailed  information  has  been  given  in  Chapter  III  as 
to  the  interfaces  of  the  manual  data  clerks  with  the  data  proces- 
sing subsystem.  In  the  actual  procedure,  two  types  of  data 
clerks  are  assumed:  (1)  clerks  for  video,  audio,  and  photo  raw 

data  extraction  and  (2)  teletype  operators  to  prepare  the  final 
punched  tape . 

5.  Engineers  and  technicians.  It  is  possible  to  identify 

three  types  of  engineers  and  technicians  necessary  for  equipment 
operation  and  maintenance  tasks:  (1)  checkout,  calibration,  and 

maintenance  of  airborne  (or  simulator)  data  acquisition  equipment, 
(2)  operation  and  maintenance  of  ground  data  processing  equipment, 
and  (3)  utilization  of  photo,  video,  and  audio  equipment  in  the 
mobile  ground  facilities.  Tasks  associated  with  the  airborne 
(or  simulator)  audio/video/photo/digital  recording  equipment  have 
been  described  in  Chapter  II  (cf..  Table  9,  page  24).  Within  the 
ground  facility,  data  processing  hardware  (cf..  Table  12,  page  35) 
is  sufficiently  extensive  to  suggest  the  necessity  for  a computer 
technician.  Finally,  the  ground  contains  a variety  of  video, 
audio,  and  photographic  equipment  (see  Appendix  A,  pages  A-17 
through  A-21)  which  must  be  maintained  and  continuously  operated. 

6.  Secretary.  A secretary  will  be  required  for  a myriad  of 
support  tasks  such  as  scheduling,  report  typing,  assisting  in 
data  collection  and  processing  when  needed,  and  so  forth. 
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Quantitative  Personnel  Requirements 


The  number  of  personnel  required  for  the  performance 
measurement  system  will  depend  upon  at  least  three  major 
parameters:  (1)  the  number  of  aircraft  (or  simulators),  flights, 

and  maneuvers  flown  for  recording  purposes  within  each  flight 
per  day,  (2)  the  specific  configuration  of  the  data  collection 
devices,  and  (3)  the  required  turn-around  time  for  data  proces- 
sing, analysis , and  interpretation.  To  provide  ranges  of 
quantitative  personnel  requirements,  variations  were  considered 
within  and  across  each  of  the  first  two  categories. 
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1*  ^ course,  very  difficult  to  predict  pr€icisely 

how  many  aircraft  might  be  expected  to  be  flown  per  day  in  future 
research  experimentation.  For  purposes  of  estimation,  a range  of 
from  one  to  four  aircraft  per  day  has  been  estimated.  Past 
experience  strongly  suggests  that  a maximum  of  two  research 
flights  per  day  per  aircraft  is  as  much  as  can  be  reasonably 
expected,  particularly  with  complex  aircraft  and  recording 
systems.  It  is,  perhaps,  better  to  estimate  that  each  aircraft 
will  give  one  valid  and  usable  recording  flight  per  day.  Within 
that  one  flight,  the  question  of  the  content  of  the  flight  in  so 
far  as  measurement  is  concerned  must  consider  the  specific 
maneuvers,  recording  techniques,  and  storage  capability  of  the 
recording  devices.  Examination  of  these  variables  from  the 
previous  and  present  phase  studies  suggest  a range  of  30-60 
minutes  of  recorded  data  pez  flight  may  be  expected.  In  the 
present  phase  report,  detailed  attention  has  been  given  to 
inflight  timing  including  off  and  on  sequencing  (cf.,  page  20; 
Figure  9,  page  22).  Efficient  recording  time  is  very  much 
influenced  by  the  presence  of  an  onboard  inflight  experimenter 
and/or  automatic  start-stop  switching. 


For  the  estimation  of  personnel  requirements,  therefore,  it 
has  been  estimated  that  the  performance  measurement  system  will 
have  to  accommodate  data  collected  from  one  to  four  aircraft  per 
day,  each  of  which  will  generate  one  recording  flight  per  day  (or 
the  equivalent  of  one  aircraft  with  two  flights  per  day  or  two 
aircraft  with  two  flights  per  day).  Range  of  data  recording 
times  are  estimated  to  vary  from  30-60  minutes. 


. ?•  Shepter  11  of  this  Phase  report  describes  in  considerable 
detail  a hybrid  performance  measurement  system  incorporating 
audio/video  recorders,  audio/digital  recorders,  and  auxiliary 
cameras  (cf..  Table  1,  page  5).  This  system  will  collect  and 
® data  in  a specific  manner,  some  of  which  will  require 
subsequent  manual  data  processing.  This  hybrid  system  has  been 
assumed  here  as  one  case,  and  is  termed  the  "complete  system"  in 
the  quantitative  personnel  requirements  estimates. 


For  the  purposes  of  comparison,  a "minimal  system"  has  also 
been  defined.  The  minimal  system  would  consist  of  a single 
audio/video  recording  system  in  the  aircraft  or  a single  cockpit 


d 


75 


r 


camera  plus  some  form  of  audio  recording.  The  minimal  system 
would,  of  course,  generate  much  less  data  than  the  complete 
system,  and,  in  addition,  would  require  far  simpler  airborne 
installations.  The  "cost"  obviously  is  in  the  completeness  and 
accuracy  of  the  raw  data  bank.  In  some  cases,  however,  aircraft 
restrictions  may  be  such  that  only  the  minimal  system  may  be 
feasible. 

3.  For  any  performance  measurement  system,  a key  determinant 
of  processing  methods  and  processing  personnel  is  the  required 
turn-around  time  for  data  processing,  analysis,  interpretation, 
and  use.  Most  such  installations  continue  to  employ  standard 
batch-processing  approaches  which  essentially  involve  collecting 
all  data  and  then  subsequently  processing  it.  This  method 
invokes  two  costs:  there  is  a considerable  time  delay  before 

data  are  available,  and,  very  often,  much  of  the' raw  data  never 
are  processed.  The  training  research  environment  is  such  that 
processed  data  should  be  made  available  as  soon  as  possible, 
particularly  if  the  data  are  to  be  used  for  knowledge  of  results 
to  instructor  and  student  pilots.  If  this  were  not  so*  data 
could  be  processed  at  leisure  with  a minimum  data  processing 
personnel  complement. 

The  present  personnel  performance  measurement  system  assumes 
that  relatively . immediate  data  processing  will  be  required.  Due 
to  the  manual  processing  steps  required  by  the  hybrid  system, 
real-time  data  processing  is  not  feasible.  However,  it  is 
assumed  that  the  data  will  be  transformed  into  computer-compatible 
form  (e.g.,  the  necessary  punched  tape)  on  the  same  day  that  they 
are  collected.  Usable  processed  data,  therefore,  will  be  avail- 
able on  the  same  day  of  the  flight  or  at  the  latest  within  24 
hours  (the  nature  of  the  available  outputs  are  shown  as  listings 
in  Figure  14,  page  48) . 

Table  22  presents  a summary  of  the  estimated  personnel 
requirements  for  the  performance  measurement  system  as  a function 
of  (1)  skill  types,  (2)  number  of  flights  per  day  from  one  to 
four,  and  (3)  either  the  complete  o.r  minimal  data  acquisition 
subsystem.  Further,  these  estimates  apply  to  the  inflight 
experimentation  case  only. 

A different  pattern  for  personnel  requirements  develops  if 
one  looks  at  the  flight  simulator  performance  measurement  require- 
ment only.  Some  assumptions  are  necessary  about  the  flight 
simulator  configuration.  First,  it  is  assumed  that  only  one 
simulator  is  available.  Thus,  simultaneous  flights  are  not 
available  as  was  the  case  with  the  aircraft.  In  this  case,  two 
simulator  "flights"  per  day  is  probably  a reasonable  average  for 
most  installations  although  past  experience  indicates  that  well- 
maintained  and  well-staffed  flight  simulators  can  accommodate  up 
to  four  "flights"  per  day.  Second,  it  is  assume'd  that  the  audio/ 
video  recorder  component  of  the  data  acquisition  subsystem  is  not 
available  as  part  of  the  simulator;  obviously,  this  is  not 
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TABLE  22 


ESTIMATED  PERSONNEL  REQUIREMENTS: 
INFLIGHT  EXPERIMENTATION 


SKILL 

CATEGORY 


COMPLETE  SYSTEM: 
FLIGHTS/DAY 


MINIMAL  SYSTEM: 
FLIGHTS/ DAY 


System  Director 
Research  Personnel 
Programmer 
Data  Clerks 

I 

(1)  Data  Tran- 
scription 

(2)  TTY  Tape  Punch 

Engineer/Technician 

(1)  A/B  Engineer 
A/B  Techincian 

(2)  Comp.  Technician 

(3)  Equip.  Technician 
Secretary 
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always  the  case.  Third,  the  audio/digital  recorder  component 
may  or  may  not  be  available  or  the  simulator  computer  may  be 
used  to  perform  as  the  digital  recorder.  Fourth,  with  regard 
to  the  data  processing  subsystem,  data  received  from  the  infiignt 
experiment  and  the  simulator  experiment  present  essentially  the 
same  entry  context. 

Table  23  presents  a summary  of  estimated  personnel  require- 
ments for  the  performance  measurement  system  as  used  in  flight 
simulator  experimentation.  The  two  data  acquisition  subsystem 
configurations  are  assumed:  complete  and  minimal.  A range  of 

from  one  to  four  "flights"  per  day  is  assumed.  With  respect  to 
skill  category,  no  requirement  exists,  obviously,  for  airborne 
installation  engineers  and  technicians;  instead,  audio/video 
technicians  are  assumed. 

It  is  reasonable  at  this  point  to  make  a number  of  statements 
about  the  tasks  of  each  skill  category  and  the  interrelationships 
between  skill  categories  from  which  the  estimates  in  Tables  22 
and  23  were  primarily  evolved: 

System  director.  In  addition  to  his  management  and  super- 
visory responsibilities , the  system  director  is  the  only  research 
specialist  for  the  lower  system  load  cases.  In  the  inflight 
experimentation  case,  he  serves  as  lead  scientist  for  the  single 
flight  per  day  either  with  complete  or  minimal  data  acquisition 
system.  For  the  flight  simulator  case  (Table  23) , he  is  the 
lead  scientist  for  both  one  and  two  simulator  "flights"  per  day. 

In  all  conditions,  he  must  serve  as  lead  scientist  for  one 
flight  per  day  whether  airborne  and  simulation. 

Research  personnel.  Quantities  of  research  personnel  vary 
between  the  inflight  and  simulator  situations.  For  inflight 
research,  it  is  standard  practice  for  each  flight  to  have  a lead 
scientist  assigned  specifically.  Sharing  across  simultaneous 
and/or  overlapping  flights  is  not  possible.  In  the  simulator, 
however,  actual  simulator  "flights"  must  be  sequential.  With 
three  or  four  simulator  flights,  only  two  lead  scientists  are 
required  (one  being  the  system  director)  with  the  only  overlap 
being  in  briefing  and  debriefing  of  subject  pilots. 

Computer  programmer.  In  the  Phase  IIIC  design  studies  it 
was  estimated  a system  programmer  might  be  desirable  during  the 
first  year  of  operation,  but  after  that  a system  load  of  at 
least  four  flights  per  day  would  be  necessary  to  justify  a 
full-time  computer  programmer. 

Data  clerks.  Whether  or  not  the  data  come  from  inflight  or 
simulator  experiments  makes  little  difference  to  the  functions 
and  tasks  of  the  data  clerks.  Data  types  and  loads  are 
essentially  equivalent  given  the  data  acquisition  configurations. 
By  its  nature,  however,  the  complete  system  requires  more  data 
processing,  and  the  increasing  loads  are  represented  in  Tables  22 
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TABLE  23 

estimated  PERSONNEL  REQUIREMENTS: 
flight  simulator  experimentation 


COMPLETE  SYSTEM: 
FLIGHTS/DAY 


SKILL 

CATEGORY 


System  Director 

Research  Personnel 

Programmer 

Data  Clerks 

Cl)  Data  Tran- 
scription 

(2)  TTY  Tape  Punch 

Engineer /Technician 

(D  Audio/Video 
C2)  Computer  Tech. 
C3)  Equip.  Tech. 
Secretary 


1 1 1 1 
0 0 1 1 
0 0 0 1 

1 1 2 2 
1 1 1 2 


MINIMAL  SYSTEM: 
FLIGHTS /DAY 


1 1 


1111 

ooii 


I 
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and  23.  With  the  minimal  system,  load  and  processing  time 
estimates  suggest  that  for  one  or  two  flights  per  day,  the  data 
transcription  clerk  can  also  generate  the  subsequent  paper 
tapes . * 

Enqineer/technicians . There  are  sharp  differences  in  the 
kinds  and  numbers  ol  engineers  and  technicians  between  the 
inflight  and  simulator  experimental  cases.  For  the  inflight  case 
(Table  22) , the  complete  system  in  the  aircraft  is  assumed  to 
require  one  engineer  for  each  set  of  two  aircraft  plus  one 
technician  per  aircraft;  these  estimates  are  based  on  actual 
experience  with  inflight  aircraft  instrumentation  situations. 

In  the  simulator  (Table  23),  a technician  is  required  to 
calibrate,  operate  and  maintain  the  audio/video  recorder.  The 
computer  technician  is  responsible  for  the  data  processing 
subsystem  in  both  cases  (plus  the  interface  to  the  audio/digital 
simulator  recorder) . With  low  flight  loads,  an  equipment, 
technician  is  not  required  with  his  duties  being  shared  either 
with  the  airborne  engineers  and  technicians  for  the  inflight 
case  and  the  audio/video  technician  for  the  simulator  situation. 

As  flights  per  day  increase,  however,  an  equipment  technician 
must  be  added  to  insure  that  all  peripheral  equipment  are 
properly  calibrated  and  operable. 

Secretary.  It  has  been  noted  that  one  of  the  tasks^of  the^ 
secretary  is  to  assist  in  audio  tape  transcription  (see  Figure  11, 
page  28).  With  the  minimal  system,  it  is  probable  that  one 
secretary  can  perform  this  subfunction.  With  higher  system  loads 
for  the  complete  system,  it  is  probable,  however,  that  an 
additional  secretary  may  be  required  to  insure  that  audio 
transcription  does  not  lag  behind  other  data  input  scheduling. 

Turn-around  time.  It  must  be  noted  again  that  the  system 
requirement  for  data  processing  turn-around  time  dictates,  to  a 
large  degree,  the  number  of  system  personnel.  If  data  processing 
can  be  relegated  completely  to  an  off-line  status  (i.e.,  at  some 
time  other  than  those  days  when  flights  are  actually  run)  a 
single  data  clerk  is  sufficient  for  all  situations,  and  even  this 
position  can  be  eliminated  if  research  and  engineering  personnel 
are  willing  to  perform  the  necessary  manual  data  processing  tasks. 
It  should  be  understood,  however,  that  this  technique  inevitably 


*It  would  appear  that  with  the  minimal  system  sampling  techniques 
can  reduce  manual  data  processing  requirements  significantly 
while  still  maintaining  satisfactory  accuracy  levels.  See: 

Isley,  R.N.,  and  Caro,  P.W. , Jr.  Use  of  time-lapse  photography 
in  flight  performance  evaluation.  Journal  of  Applied  Psychology, 
1970,  5_4  (1)  , 72-76. 

♦♦Although  it  should  be  noted  that  practice  varies  widely;  Table 
22  probably  represents  the  minimal  case. 
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means  a very  considerable  time  delay  between  data  collection 
and  data  in  a usable  format.  It  is  difficult  to  see  how  the 
training  research  context  can  afford  this  particular  kind  of 
delay  if  modern  training  concepts  such  as  adaptive  training  and 
knowledge  of  results  for  learner-centered  training  systems  are 
to  be  implemented. 
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V.  IMPLEMENTATION  PLAN 


System  Acquisition  Methods 


The  analysis  to  this  point  has  described  all  of  the  elements 
necessary  for  a potential  performance  measurement  system.  The 
next  step  is  an  implementation  plan  for  system  requisition.  Many 
approaches  are  possible,  but  the  most  effective  method  appears  to 
be  one  based  on  Air  Force  Systems  Command  Manual  AFSCM  375-5: 
System  Engineering  Management  Procedures.  Figure  22  diagrams 
the  proposed  measurement  system  implementation  plan.  Figure  23 
presents  a coordinated  schedule  for  system  implementation. 


As  shown  in  both  figures,  five  major  steps  are  required: 


Selection  of  a system  integration  contractor; 


Completion  of  preliminary  detail  system  and 
subsystem  design; 


Selection  of  the  final  system  design  with 
appropriate  Category  I testing; 


Procurement  of  system  hardware  and  system 
integration;  and 


Completion  of  final  system  tests  resulting 
in  system  turnover  to  the  Air  Force. 


System  Integration  Contractor 


The  final  performance  measurement  system  will  be  a;  collection 
of  a variety  of  hardware  and  software  components.  With  respect 
to  hardware  subsystems,  very  extensive  use  is  expected  of  off-the- 
sheli  components  (see  Appendix  A) . At  this  stage  in  any  system 
acquisition,  prior  experience  strongly  suggests  the  desirability 
of  a system  integration  contractor  working  under  the  direction  of 
the  Air  Force. 


To  use  the  AFSCM  375-5  methodology,  it  is  assumed  that 
sufficient  technical  inputs  are  available  to  establish  a satis- 
factory Statement  of  Work  for  the  development  contract  (Figure  22; 
Step  1.1).  The  results  of  the  present  study  are  considered  to 
be  the  basic  and  necessary  technical  data  to  execute  this  step. 


Design  Review 


However,  with  the  normal  time  span  that  will  occur  before  a 
development  contract  is  initiated,  technological  advances  may  be 
expected.  Therefore,  preliminary  detail  design  for  the  final 
system  is  necessary  (Figure  22;  Step  2.1)  to  update  the  inpxit 
data.  In  Figure  23,  a two-month  period  has  been  allocated  for 
this  step,  a time  period  which  is  very  brief  and  which  may  have 
to  be  extended. 
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numbers  refer  to  the  applicable 
in  the  AFSCK  375~5  methodology 
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Figure  22  - (continued) 
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To  insure  that  development  is  proceeding  properly * a prelim- 
inary design  review  is  scheduled  (Figure  22;  Step  2.2).  This 
review  is  a control  step  to  provide  direction  to  the  system 
integration  contractor  by  the  Air  Force. 

Final  Design  and  Test 

Eight  critical  items  have  been  selected  for  emphasis  in 
final  system  design  (Figure  22;  Steps  3.1  through  3.8).  Three 
of  these  items  may  be  pointed  out  for  particular  emphasis: 

1.  The  earliest  possible  aircraft  bailment  (Step  3.1)  is 
recommended.  The  necessary  modifications  for  in-flight  perform- 
ance measurement  equipment  appear  to  be  extremely  time-consuming, 
based  on  the  data  collected  in  the  study  survey. 

2.  Detailed  design  of  the  final  computer  software  programs 
(Step  3.2)  is  the  longest  single  item  (see  Figure  23).  It  is 
very  difficult  to  predict  how  long  will  be  required  for  program 
development,  de-bugging,  checkout,  and  test.  Past  experience 
suggests  that  this  step  may  be  the  critical  path  in  the  acquisi- 
tion network. 

3.  Some  selected  hardware  procurement  (Step  3.4)  may  be 
required,  primarily  for  test  and  analysis  purposes.  A specific 
case  in  point  is  the  video  recorders  where  basic  questions  exist 
as  to  their  capability  in  the  in-flight  environment.  If  possible, 
selection  of  the  computer  would  be  desirable  at  this  point  although 
it  is  not  essential  for  development. 

At  the  end  of  this  phase,  final  detailed  specifications 
(Step  3.10)  should  be  available  for  the  critical  design  review 
(Step  3.11)  after  which  the  final  performance  measurement  system 
is  frozen. 

System  Procurement  and  Integration 

All  subsystems  are  now  procured,  and  system  integration  is 
accomplished.  As  shown  in  Figure  22,  the  airborne  (Step  4.6)  and 
ground  (Step  4.5)  subsystems  have  been  kept  seperate  under  the 
possibility  that  one  and  not  both  might  be  acquired.  This 
separation  is  not  recommended,  but  practical  considerations  may 
intervene. 

Final  System  Tests  and  Turnover 

Category  II  tests  are  shown  for  the  ground  data  processing 
subsystem  (Step  5.1)  and  the  in-flight  data  acquisition  subsystem 
(Step  5.2).  It  is  anticipated  that  the  technical  approval 
demonstration  (Step  5.3)  will,  in  fact,  be  the  execution  of  a 
complete  research  experiment  conducted  jointly  by  the  system 
integration  contractor  and  Air  Force  technical  personnel.  With 
this  techniques,  total  system  testing  can  be  achieved;  usable 
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research  data  will  be  collected;  and  Air  Force  technical  personnel 
wiH  be  provided  training  on  the  system  under  the  most  realistic 
of  research  environments. 

Implementation  Schedule 

As  noted.  Figure  23  presents  a time  schedule  covering  all 
the  steps  in  the  implementation  plan.  As  may  be  seen  in  that 
Figure,  a 28-month  program  is  anticipated.  Based  on  past  experience 
and  the  information  collected  during  this  study,  it  is  believed 
that  this  is  a realistic  system  acquisition  schedule  for  the 
performance  measurement  system. 


i 
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appendix  a.  equipment  survey 


Introduction 

This  section  presents  a tentative  survey  and  specification 

of  equipment  and  instrumentation  re9uir?^  £“t“|^'n*erived  from 
CCTS  performance  measurement  program.  , th  evalu- 

manufacturer 1 s published 

ation  of  the  equipment  presented  (and  other  alternative  eqp 

Sr^L^ir^o^f^rS  s -Equipment 

necessary^to  res tric^the^surve^of Sequipment^ton" representative" 

samples  of  the  various  areas. 

Appendix  A has  three  major  sections.  Section  One  addresses 

equip‘ 

discussed^theSpost^f light  ground^ebriefing  facilities^and^  ^^^^ 

associated  equipment.  Section  inre®  a . . t d for  detailed 

facilities  and  equipment  requirements  anticip 
analysis  and  evaluation  of  the  collected  data. 

Section  One:  Airborne  Equipment  Requirements 

Fiaure  1 presents  an  overview  of  the  required  airborne  data 

sis hfa^M 

supporting  timing  and  initiating  devices  as  appropriate. 

Andio/Video  Recorders.  Previous  attempts  to  use  relatively 

inexpensive  of f-the-sheif” video 

^“ilionfo?  -« 

demonstrate,  however,  that  Implications . 

could  operate  in  an  acceptable  manner  in  airoorne  aPP 

A review  of  inexpensive  commercial ^ 9/^°^ 
(Table  1)  and  cameras  (Tables  2 . -ILm!9  twice^he  resolution 

n5?-5?0nUnesf  o?\heUeq^pment  used  In  Ihe  above  mentioned 

SSSS^-^-SS  fnf  reroIdels^oT^f  ^ the 
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TABLE  1 VIDEO  RECORDER  SURVEY 


VIDEO  CAMERA  SURVEY 
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TABLE  3 

SPECIAL  EQUIPMENT 


SPECIAL  EFFECTS  GENERATORS: 

Panasonic  Model  WJ-540 

Craig  Model  9823 
Model  9824 
Model  9825 

SEQUENTIAL  SWITCHES: 

Craig  Model  9826 


$325 

300 

350 

350 


200 


w_c  restricted  to  equipment  with  moderately  high  reso- 
lution, small  packaging  and  modest  initial  costs.  Within  these 
constraints,  a number  of  alternative  cameras  and  recorders  are 
commercially  available  and  should  be  examined  more  closely. 


In  securing  equipment  for  the  present  application,  two 

approaches  are  possible:  arbitrary  selection  of  » “^eral 

a video  recorder  based  on  the  survey  data,  or  S e£  prior 

rampras  and  se\eral  recorders  and  laboratory  testing  them  pno 

to  installation.  Since  many  pieces  of  this  chh i thS 

been  field  tested,  it  is  felt  that  the  latter  approach  is 

most  reasonable  in  the  long  run.  ^ ,serl«  s^P^e9Sccept- 
of  equipments  selected  should  provide  evidence  of  the  accept 

ability^nd  airworthiness  of  various  combinations  of  cam, “suit 
rppnrders  prior  to  installation  in  aircraft.  This  would  result 

in  considerable  savings  in  terms  of  ^^"^“^inc^ed  in 
time.  These  savings  in  turn  should  offset  the  cost 
securing  several  different  cameras  and  recorders. 


vidpo  Recorder  Testing.  A tentative  list  of  c^ras,  ^video 
recorders'  and  other  assorted  equipment  to  °s 

procurement  and  should  not  be  construed  as  ^"^g^^stem" 
dependent  systems.  The  items  purchased  for  the  fas ic  system 
woSld  be  combined  with  and  tested  with  items  purchased  for  the 
"intermediate”  and  the  "high  resolution  system.. 


The  test  pain  would  be  to  simply  combine  each  video  camera 

with  ea^h  video  recorder  and  perform  a s^^a^0^P^e  ?ests 
obtain  samples  of  video  tape  from  each  combination.  The  tes 
should  include  exposures  to  low,  moderate,  high  and  r p Y 
chanainq  illumination  conditions,  several  scene  panning  rates, 
and  levlril  target  backgrounds.  These  test  samples  would  be 
examined  and  evaluated  by  the  video  equipment  integration  con 
tractor  for  image  quality  and  equipment  serviceability.  pro 
iections  could  then  be  made  as  to  the  optimum  combination 
components  for  installation  into  the  test  aircraft.  Sample 
testing  combinations  are  presented  in  Figure  i. 


other  Equipment.  In  additiontoevaluating  the  recordingjquip- 
ment  tests  should  be  conducted  to  determine  the  effective  e 
^iaXfof  .using  a special 

^fiSlir-coJdfng1"?^' 'iSm'SSfiient  cameras  for  differing 
lengths  of  time  on  some  film).  Likewise,  the  Cr aig  automatic 
light  doucher  (which  reduces  light  entering  cancra  if  rapidly 
• increaseing  illumination  exceeds  the  camera's  automatic  adjust 
ment  capability)  should  be  evaluated. 
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TABLE  4 


TENTATIVE  EQUIPMENT  RECOMMENDATIONS 


HIGH  RESOLUTION  SYSTEM  -V 


HIGH  RESOULTION  SYSTEM  -2 


Figure  2.  Tentative  Test  Configurations 


ECHO 

WRR-211 

RECORDER 


ECHO 

WRR-211 

RECORDER 


(Continued) 
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has 

cameras3^ two^or  «^t  i» 

a manner  which  would  optimise  bf eS moS  £ o maintain  the 

coverage  Each  camera  would  be  then  be  connected 

optimum  field  of  coverage.  i,i  effects  qenerator 

^SgSgsS-safiS^ 

Sd^sls  external  *«., 

or  event  specific  filming . A camera  can  be  setup  with a data 
chamber  which  can  ^“^“V^ioSSSc  scanning  device  can  be 
included^so^tha^a^single” camera  can  cover  a target  area  larger 
than  its  normal  field  of  view. 

With  the  enormous  variety  of  photographic  equipment  available, 
it  isWdi«iceuietnto  specify  a given  camera  for  a given  application 

I^riL?fifn^ef^  correctors  MM  earner  as  for  specific 
instai  . SDecific  aircraft;  several  manufacturers  build 

^eifsThLrarrfuy^daptab  e t< . several  systems  - and  a 
number  of  manufacturers  P^uce  commercially  ava  technically 
relatively  inexpensive)  camera  systems  wni  been  utilized 

capable  of  airborne  application,  but  whic  establish  the 

fo?  this  purpose.  The  problem,  ccnsequently,  ^^establish^  a 

technical  capabilities  require  b' summarizes  data  for  a number 

cost-effectiveness  tradeoff..  Table  b s“i4  . suxnmary  a 

of  currently  available  Ph°t°9taphic  systems^  In^this^^ 

number  of  commercially  avail  > PTable  6yiists  suamary  data 

for  current ly^available1  supporting  equipment  for  airbonre  photo- 

graphic  applications. 

Tentative  candidates  for  the i;*f“1JJb^i|Bted  ““ 

externally  oriented  cameras  are  listed  in  Table  . 
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TABLE  7.  CANDIDATE  CAMERAS 


EXTERNAL 

INTERNAL 

Photosonic  KB-25A 

Photosonic  1VN 

or 

or 

Milligan  DBM-3C 

Visual  Instrument 
SP-1 

Cockpit  Voice  Recorder^.  It  is  anticipated  that  an  additional 
voice  recording  system  will  be  required  for  (a)  backup  in  the  event 
of  audio-visual  recording  system  failure,  arid  (b)  a primary  voice 
recording  system  in  the  event  the  audio-visual  system  proves  un- 
acceptable. » 

A number  of  voice  recorders  are  produced  specifically  for 
airborne  application.  A number  of  commercially  available  recorders, 
however,  appear  to  offer  sufficient  potential  for  meeting  present 
requirements  at  a considerable  reduction  in  procurement  costs. 

These  recorders  are  summarized  in  Table  8. 

Based  on  the  manufacturer's  published  data,  the  Sony  TC-110A 
is  recommended  for  consideration  and  further  evaluation. 

Airborne  Digital  Recorders.  An  airborne  digital  recording 
system  is  required  for  measurement  of  instrumentation  and  aircraft 
outputs  during  critical  periods  of  the  student/pilot's  flight. 

A number  of  such  systems  are  available.  Several  of  these  are 
summarized  in  Table  9.  For  the  present  application,  the  Incra- 
Data  Mark  II  appears  to  offer  the  most  potential. 

Section  Two;  Post  Flight-Debriefing  Requirements 

This  section  addresses  the  facilities  required  for  housing 
the  post-flight  debriefing  rooms  and  equipment  as  well  as  the  data 
processing  equipment  (discussed  in  Section  Three) . In  addition 
to  the  ground  mobile  facilities,  the  required  debriefing  equip- 
ment will  be  summarized. 

Ground  Mobile  Facilities.  A ground  mobile  debriefing  facility 
and  data  processing  center  are  required  for  this  program.  These 
facilities  could  be  mounted  in  several  truck  drawn  trailers  or 
in  one  or  two  large  mobile  home  type  of  trailer  devoid  of  interior 
partions  and  furnishings  (with  the  exception  of  restroom  facilities). 
'It  is  felt  that  the  latter  type  of  vehicle  would  be  the  most 
flexible  in  terms  of  equipment  configurations,  personnel  organ- 
ization, and  functional  utilization  of  available  space. 
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TABLE  8.  COCKPIT  VOICE  RECORDER  SURVEY 
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Rasio  Requirements.  The  facilities  ate  required  to  houses 
(a)  post-flight  debriefing  rooms. 

$ Limited°office^space^for^  technical  personnel. 

(d)  Work  area  for  equipment  technicians. 

(e)  Work  area  for  supporting  secretarial  per- 
sonnel. , , . 

(f)  Storage  area  for  records  and  data. 

(g)  Restroom  facilities. 

'Tmn^f.ive  Configurations.  These  facilities  can  bejioused 

in  either  one  12-foot  wide  W 60-foot  long  mobile -table  in' two 

vehicle  (somewhat  cramped,  however)  °*  ustng 

such  vehicles.  Figure  3 sugges  P ..  little  room  for 

two  vehicles.  Single-unit  configurations  permit  little  room 

offices. 

annroximate  Cost.  The  mobile  vehicles  suggested  in  Figure  3 

Sts" ?h:PheavyCdutr£lSor?ng1anI  trame  required  to  accomodate 
the  data  processing  equipment. 

Requirements.  The  basic  equipment  required  for 
post-flight  debriefing  consists  of*,. 

(a)  Dictation/tape  recorder  equipment. 

(b)  16mm  (or  8ram)  file  viewer/printer. 

(c>  Video  tape  monitor. 

(d)  Video  tape  playback  unit. 

one  of  each  of  the  above  pieces  of  equipment  would  be  required- 
for  each  °of  thfdebrief  ing  -oL  used. 

two  debriefing  rooms  are  anticipated.  A tentative  con  ^ 
is  presented  in  Figure  4. 

Video  Monitors.  Table  10  suimarize^taeK^cal_specificatrons 

, — g amp le  of  the  vide-erTnonitors  commercially  ava^i 
and  prices  for  a sample  ot  f that  a 12-inch  monitor 

able.  For  the  present  prpgreiC  it  is  felt  that  a iz 1 

should  be  suf  f iciejit-  fSrreviewing  video  tapes.  Either  y 

or  the  Cor^s^M^'  monitor  should  be  considered. 

"^video  Playback  Equipment.  Two  (or  more)  video  recorders  . 

ers)?  For  the  present  application,  the  Shibaden  model  tVC-250  or 

the  Javalin  model  X-400TVR  should  be  considered. 
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Figure  4.  Tentative  Debriefing  Room. 
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MOBILE  FACILITY 
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- Dictaphone 

- 16mm  (8mm)  Viewer/Printer 

- Video  Monitor 

- Video  Playback 
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Film  Viewer/Printer.  A number  of  manufacturers  produce 
8mm,  16mm,  or  35mm  film  viewers  and  frame  printers.  The  3-M 
Company  Model  "400"  reader-printer  is  a proven  product  and  is 
recommended  for  the  present  application.  This  model  sells  for 
approximately  $1,500  for  the  motorized  version  and  accepts 
either  16mm  or  35mm  film. 

Dictation  Equipment.  The  dictation  or  tape  recording 
equipment  required  for  verbal  data  collection  purposes  need  not 
be  of  exceptionally  high  quality  and  can,  therefore,  be  selected 
from  among  the  large  number  of  inexpensive  recording  systems 
currently  available.  Table  11  summarizes  the  specifications  for 
a number  of  these  recorders.  For  the  present  application,  the 
Sony  TC-40  tape  recorder  appears  to  meet  all  requirements. 

* 

Section  Three:  Data  Processing  Equipment 

This  section  briefly  outlines  some  of  the  commercially 
available  equipment  meeting  the  general  requirements  established 
in  the  Data  Processing  Subsystem  (discussed  in  Section  III  of 
text).  Recommendations  made  in  this  section  are  tentative  and 
made  for  illustrative  purposes  only.  A more  exhaustive  specifica- 
tion of  requirements  would  be  required  prior  to  any  final 
selection  of  components . 

Equipment  Required.  Figure  5 presents  an  overview  of  the 
data  processing  and  peripheral  equipment  required  for  the  present 
program.  The  equipment  consists  of: 

(a)  General  purpose  digital  computer. 

(b)  Input/output  devices  including: 

Magnetic  Tape  Units 
Card  Reader 

Paper  Tape  Punch/Reader 

Line  Printer/Plotter 

Teletype 

CRT  Data  Viewer 

Disc  Storage 

Other  Data  Viewing  Equipment 

(c)  Appropriate  data  conversion  and  control  equipment. 

Diqital  Computer.  After  an  initial  survey  and  screening  of 
currently  available  digital  computers,  the  twenty-five  computers 
listed  in  Table  12  appear  to  meet  the  technical  requirement  of 
the  present  program. . It  is  observed  that  the  range  of  capabili- 
ties and  cost  of  these  remaining  systems  is  great.  Even  within 
one  computer  model,  capability  and  procurement  costs  are  quite 
variable.  The  PDP  model  15  or  the  Raytheon  Model  704  (or 
equivalent  alternatives),  however,  represent  good  potential 
candidates  for  the  present  program. 
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TnrmWoui-Dut  Devices.  An  infinite  variety  of  peripheral 
equipment  exists  tor  curTently  available  digital  computers, 
following  surveys  again  suggest  representative  samples  ot 
available  equipment.  One  particular  tradenameandmodel  for 
each  piece  of  equipment  has  been  selected  for  the  Purpose  or 
illustration.  This  does  not  suggest,  however,  that  other  m 
could  not  perform  equally  well. 

Maanetic  Tape  I/O  Units.  Table  13  summarizes  specif i- 
cations  for  l number  Sf  oommerHJlly  available  magnetic  tape 
units.  The  Ampex  series  is  representative  for  the  present 

program. 

Disc  Recorders.  Table  14  presents  a brief  survey^of 
candidate  disc  recording  units.  Of  the, units  listed,  the  ISS 
moving  head  recover  unit  appears  to  offer  the  most  “^blllty 
and  flexibility  of  application.  It  is  suggested  for  the  present 

program. 

Panpr  Tape  I/O  Units.  Table  15  surveys  a number  of 
available  paper  tape  punching" and  reading  units.  The  Bunker- 
Ra^SelPmappears  to  heritable  for  the  present  program. 

Data  Terminals.  Table  16  summarizes  a number  of 
currently  available  dat5T  terminals.  The  ITEL  model  1051  is 
Zst  flexible  of  these  units  and  is  suggested  as  a candidate. 

rnmnill-pr  Displays.  Table  17  summarizes  a number  of 
commerc i a 1 ly™ aval  1 ab ie^d a^ta  display  systems.  The  Infoton  display 
U™  p«ven  and  acceptable  model  for  the  present  program. 

,,,Qnfafivp  cnnf iauration.  Figure  6 presents  a typical 

Sssss-ilh 

Cost.  It  is  impossible  to  specify  cost  of  the  data  pro ces- 
^^iSor^r^fciarl^t^rlsL^^r^^r^  cpj  ons  f°rum_ 

tables^are "rough*1^ s timates " ' and  ^^“lefieft  the  base 
coJt  of b the  item  described.  It  does  not  include  ^sta^;*!;lon 
cost,  supporting  hardware  or  possible  modifications . More 
definitive  cost  estimates  can  only  be  derived  fr 
negotiations  with  manufacturers . 

conversion  and  Control  Equipment.  At  this  point,  it  would 
desIgn!dOTf“dthrmosi  parlfs^in^y  "r>e  peripheral  and 

ndSitr-S^a^arft^fler^ 

interface  many  of  the  component  subsystems  discussed  above. 
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